python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3338443 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2860281'] with mtr_portfolio_ids : ['23003831'] and first list_photo_ids : [] new path : /proc/3338443/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 6 ; length of list_pids : 6 ; length of list_args : 6 time to download the photos : 1.255049705505371 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu May 15 13:00:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3169 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-15 13:00:33.775374: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-15 13:00:33.803108: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-15 13:00:33.804775: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fec70000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-15 13:00:33.804828: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-15 13:00:33.807839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-15 13:00:34.069744: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1aa10bb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-15 13:00:34.069795: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-15 13:00:34.070458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-15 13:00:34.071837: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-15 13:00:34.077294: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-15 13:00:34.081927: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-15 13:00:34.083243: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-15 13:00:34.087269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-15 13:00:34.090006: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-15 13:00:34.099678: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-15 13:00:34.100866: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-15 13:00:34.100969: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-15 13:00:34.101540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-15 13:00:34.101556: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-15 13:00:34.101565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-15 13:00:34.102916: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2717 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-15 13:00:34.467946: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-15 13:00:34.468063: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-15 13:00:34.468082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-15 13:00:34.468100: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-15 13:00:34.468117: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-15 13:00:34.468134: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-15 13:00:34.468150: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-15 13:00:34.468167: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-15 13:00:34.468970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-15 13:00:34.470160: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-15 13:00:34.470197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-15 13:00:34.470212: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-15 13:00:34.470226: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-15 13:00:34.470240: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-15 13:00:34.470254: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-15 13:00:34.470268: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-15 13:00:34.470282: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-15 13:00:34.471110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-15 13:00:34.471152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-15 13:00:34.471161: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-15 13:00:34.471169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-15 13:00:34.472072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2717 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-15 13:00:44.442820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-15 13:00:44.720030: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-15 13:00:46.292927: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.293015: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.293824: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.293850: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.301470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.301523: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.302081: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.302097: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.309645: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.309693: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.310433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.310453: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.342359: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.342400: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.343075: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.343094: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.348937: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.348958: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.349539: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.349555: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-15 13:00:46.383699: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.384421: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.386308: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.386942: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.431167: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.431796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.434095: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.434709: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.465192: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.465811: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.467561: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.468166: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.475031: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.475673: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.477667: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.478280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.484750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.485423: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.487030: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.487633: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.515202: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.515844: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.516447: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.517048: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.521242: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.521872: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.538447: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.539128: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.539735: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.540340: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.553672: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.554325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.555028: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.555635: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.560179: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.560818: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.565630: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.566284: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.578723: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.579390: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.583782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.584437: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.585052: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.585665: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.586691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.587353: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.598215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.598873: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.599515: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.600118: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.600720: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.601320: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.617746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.618748: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.673965: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.674619: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.682644: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.683309: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.701190: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.701842: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.702480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.703119: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.707426: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.708045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.708664: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.709266: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.710983: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.711006: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-15 13:00:46.720753: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.721384: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.730661: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.731305: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.731939: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.732539: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.733146: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-15 13:00:46.733743: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.15G (2314076160 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 6 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 43 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 44 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 36 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 41 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 31 Detection mask done ! Trying to reset tf kernel 3339412 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1757 tf kernel not reseted sub process len(results) : 6 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 6 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2950 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0009102821350097656 nb_pixel_total : 24808 time to create 1 rle with old method : 0.03049302101135254 length of segment : 259 time for calcul the mask position with numpy : 0.0006692409515380859 nb_pixel_total : 38279 time to create 1 rle with old method : 0.04358625411987305 length of segment : 221 time for calcul the mask position with numpy : 0.44064807891845703 nb_pixel_total : 4289472 time to create 1 rle with new method : 0.31464147567749023 length of segment : 1744 time for calcul the mask position with numpy : 0.0007910728454589844 nb_pixel_total : 47618 time to create 1 rle with old method : 0.0639793872833252 length of segment : 242 time for calcul the mask position with numpy : 0.0008037090301513672 nb_pixel_total : 49683 time to create 1 rle with old method : 0.05647611618041992 length of segment : 199 time for calcul the mask position with numpy : 0.00018906593322753906 nb_pixel_total : 7463 time to create 1 rle with old method : 0.008835077285766602 length of segment : 100 time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 8578 time to create 1 rle with old method : 0.010318756103515625 length of segment : 110 time for calcul the mask position with numpy : 0.002442598342895508 nb_pixel_total : 112557 time to create 1 rle with old method : 0.12609434127807617 length of segment : 404 time for calcul the mask position with numpy : 0.0008931159973144531 nb_pixel_total : 45550 time to create 1 rle with old method : 0.051610469818115234 length of segment : 431 time for calcul the mask position with numpy : 0.001056671142578125 nb_pixel_total : 52383 time to create 1 rle with old method : 0.08055758476257324 length of segment : 195 time for calcul the mask position with numpy : 0.0004737377166748047 nb_pixel_total : 14939 time to create 1 rle with old method : 0.0171205997467041 length of segment : 185 time for calcul the mask position with numpy : 0.0005078315734863281 nb_pixel_total : 22439 time to create 1 rle with old method : 0.026187896728515625 length of segment : 145 time for calcul the mask position with numpy : 0.002626657485961914 nb_pixel_total : 180090 time to create 1 rle with new method : 0.009726285934448242 length of segment : 544 time for calcul the mask position with numpy : 0.0003962516784667969 nb_pixel_total : 20482 time to create 1 rle with old method : 0.023984193801879883 length of segment : 109 time for calcul the mask position with numpy : 0.0021691322326660156 nb_pixel_total : 122243 time to create 1 rle with old method : 0.13738465309143066 length of segment : 417 time for calcul the mask position with numpy : 0.00025844573974609375 nb_pixel_total : 7501 time to create 1 rle with old method : 0.008992195129394531 length of segment : 94 time for calcul the mask position with numpy : 0.0004150867462158203 nb_pixel_total : 19777 time to create 1 rle with old method : 0.023159265518188477 length of segment : 167 time for calcul the mask position with numpy : 0.00099945068359375 nb_pixel_total : 57918 time to create 1 rle with old method : 0.06588935852050781 length of segment : 336 time for calcul the mask position with numpy : 0.00039839744567871094 nb_pixel_total : 17570 time to create 1 rle with old method : 0.0213468074798584 length of segment : 144 time for calcul the mask position with numpy : 0.004622697830200195 nb_pixel_total : 154899 time to create 1 rle with new method : 0.01534891128540039 length of segment : 806 time for calcul the mask position with numpy : 0.0011866092681884766 nb_pixel_total : 25448 time to create 1 rle with old method : 0.029150009155273438 length of segment : 298 time for calcul the mask position with numpy : 0.0013761520385742188 nb_pixel_total : 27646 time to create 1 rle with old method : 0.032088518142700195 length of segment : 203 time for calcul the mask position with numpy : 0.0020906925201416016 nb_pixel_total : 43058 time to create 1 rle with old method : 0.05881071090698242 length of segment : 316 time for calcul the mask position with numpy : 0.0006723403930664062 nb_pixel_total : 10737 time to create 1 rle with old method : 0.01693415641784668 length of segment : 105 time for calcul the mask position with numpy : 0.0038781166076660156 nb_pixel_total : 90352 time to create 1 rle with old method : 0.11162114143371582 length of segment : 658 time for calcul the mask position with numpy : 0.004467964172363281 nb_pixel_total : 148445 time to create 1 rle with old method : 0.16693401336669922 length of segment : 386 time for calcul the mask position with numpy : 0.0009992122650146484 nb_pixel_total : 18390 time to create 1 rle with old method : 0.0227508544921875 length of segment : 156 time for calcul the mask position with numpy : 0.0017230510711669922 nb_pixel_total : 33896 time to create 1 rle with old method : 0.038523197174072266 length of segment : 281 time for calcul the mask position with numpy : 0.0020432472229003906 nb_pixel_total : 52160 time to create 1 rle with old method : 0.06335854530334473 length of segment : 322 time for calcul the mask position with numpy : 0.0008809566497802734 nb_pixel_total : 12268 time to create 1 rle with old method : 0.014272689819335938 length of segment : 198 time for calcul the mask position with numpy : 0.0008482933044433594 nb_pixel_total : 14851 time to create 1 rle with old method : 0.01752305030822754 length of segment : 179 time for calcul the mask position with numpy : 0.0026853084564208984 nb_pixel_total : 65331 time to create 1 rle with old method : 0.07384276390075684 length of segment : 485 time for calcul the mask position with numpy : 0.0028808116912841797 nb_pixel_total : 74991 time to create 1 rle with old method : 0.08484792709350586 length of segment : 267 time for calcul the mask position with numpy : 0.00177764892578125 nb_pixel_total : 50312 time to create 1 rle with old method : 0.05817675590515137 length of segment : 196 time for calcul the mask position with numpy : 0.0005540847778320312 nb_pixel_total : 8307 time to create 1 rle with old method : 0.010978221893310547 length of segment : 80 time for calcul the mask position with numpy : 0.0004172325134277344 nb_pixel_total : 8969 time to create 1 rle with old method : 0.01162099838256836 length of segment : 92 time for calcul the mask position with numpy : 0.0007622241973876953 nb_pixel_total : 18844 time to create 1 rle with old method : 0.022261381149291992 length of segment : 159 time for calcul the mask position with numpy : 0.0033903121948242188 nb_pixel_total : 24838 time to create 1 rle with old method : 0.028566837310791016 length of segment : 435 time for calcul the mask position with numpy : 0.0015790462493896484 nb_pixel_total : 33473 time to create 1 rle with old method : 0.051952362060546875 length of segment : 212 time for calcul the mask position with numpy : 0.004181861877441406 nb_pixel_total : 79083 time to create 1 rle with old method : 0.09572815895080566 length of segment : 362 time for calcul the mask position with numpy : 0.0011179447174072266 nb_pixel_total : 28042 time to create 1 rle with old method : 0.032103538513183594 length of segment : 176 time for calcul the mask position with numpy : 0.0011031627655029297 nb_pixel_total : 18605 time to create 1 rle with old method : 0.0220186710357666 length of segment : 192 time for calcul the mask position with numpy : 0.00112152099609375 nb_pixel_total : 25632 time to create 1 rle with old method : 0.0293729305267334 length of segment : 193 time for calcul the mask position with numpy : 0.000843048095703125 nb_pixel_total : 20895 time to create 1 rle with old method : 0.023992538452148438 length of segment : 182 time for calcul the mask position with numpy : 0.0003879070281982422 nb_pixel_total : 18792 time to create 1 rle with old method : 0.021852731704711914 length of segment : 159 time for calcul the mask position with numpy : 0.0011560916900634766 nb_pixel_total : 29619 time to create 1 rle with old method : 0.03401803970336914 length of segment : 195 time for calcul the mask position with numpy : 0.0006694793701171875 nb_pixel_total : 13149 time to create 1 rle with old method : 0.01535177230834961 length of segment : 163 time for calcul the mask position with numpy : 0.0012824535369873047 nb_pixel_total : 41214 time to create 1 rle with old method : 0.05070996284484863 length of segment : 270 time for calcul the mask position with numpy : 0.0009019374847412109 nb_pixel_total : 22242 time to create 1 rle with old method : 0.027266502380371094 length of segment : 197 time for calcul the mask position with numpy : 0.0013697147369384766 nb_pixel_total : 34253 time to create 1 rle with old method : 0.03998756408691406 length of segment : 284 time for calcul the mask position with numpy : 0.0012390613555908203 nb_pixel_total : 25447 time to create 1 rle with old method : 0.02897191047668457 length of segment : 238 time for calcul the mask position with numpy : 0.0006768703460693359 nb_pixel_total : 16381 time to create 1 rle with old method : 0.027498483657836914 length of segment : 92 time for calcul the mask position with numpy : 0.0005979537963867188 nb_pixel_total : 14012 time to create 1 rle with old method : 0.021937847137451172 length of segment : 162 time for calcul the mask position with numpy : 0.002160787582397461 nb_pixel_total : 52974 time to create 1 rle with old method : 0.06169295310974121 length of segment : 290 time for calcul the mask position with numpy : 0.0017809867858886719 nb_pixel_total : 31369 time to create 1 rle with old method : 0.046114444732666016 length of segment : 186 time for calcul the mask position with numpy : 0.0025663375854492188 nb_pixel_total : 74693 time to create 1 rle with old method : 0.08507084846496582 length of segment : 264 time for calcul the mask position with numpy : 0.0012347698211669922 nb_pixel_total : 15706 time to create 1 rle with old method : 0.01900768280029297 length of segment : 192 time for calcul the mask position with numpy : 0.003113985061645508 nb_pixel_total : 54288 time to create 1 rle with old method : 0.06146383285522461 length of segment : 873 time for calcul the mask position with numpy : 0.0028786659240722656 nb_pixel_total : 71792 time to create 1 rle with old method : 0.12859511375427246 length of segment : 314 time for calcul the mask position with numpy : 0.0007777214050292969 nb_pixel_total : 7334 time to create 1 rle with old method : 0.012745141983032227 length of segment : 113 time for calcul the mask position with numpy : 0.005093097686767578 nb_pixel_total : 58912 time to create 1 rle with old method : 0.1164543628692627 length of segment : 185 time for calcul the mask position with numpy : 0.003040313720703125 nb_pixel_total : 58817 time to create 1 rle with old method : 0.07429218292236328 length of segment : 343 time for calcul the mask position with numpy : 0.0020089149475097656 nb_pixel_total : 46620 time to create 1 rle with old method : 0.05514335632324219 length of segment : 333 time for calcul the mask position with numpy : 0.0011250972747802734 nb_pixel_total : 23818 time to create 1 rle with old method : 0.027599573135375977 length of segment : 242 time for calcul the mask position with numpy : 0.0026476383209228516 nb_pixel_total : 54516 time to create 1 rle with old method : 0.06561875343322754 length of segment : 309 time for calcul the mask position with numpy : 0.0027837753295898438 nb_pixel_total : 19104 time to create 1 rle with old method : 0.025693655014038086 length of segment : 334 time for calcul the mask position with numpy : 0.001501321792602539 nb_pixel_total : 37213 time to create 1 rle with old method : 0.045278072357177734 length of segment : 296 time for calcul the mask position with numpy : 0.0011157989501953125 nb_pixel_total : 22282 time to create 1 rle with old method : 0.025562524795532227 length of segment : 191 time for calcul the mask position with numpy : 0.0012354850769042969 nb_pixel_total : 32317 time to create 1 rle with old method : 0.03776979446411133 length of segment : 149 time for calcul the mask position with numpy : 0.0025739669799804688 nb_pixel_total : 59691 time to create 1 rle with old method : 0.0699012279510498 length of segment : 385 time for calcul the mask position with numpy : 0.001004934310913086 nb_pixel_total : 39340 time to create 1 rle with old method : 0.04458189010620117 length of segment : 293 time for calcul the mask position with numpy : 0.0057985782623291016 nb_pixel_total : 150631 time to create 1 rle with new method : 0.008689641952514648 length of segment : 522 time for calcul the mask position with numpy : 0.0012764930725097656 nb_pixel_total : 25846 time to create 1 rle with old method : 0.031694650650024414 length of segment : 202 time for calcul the mask position with numpy : 0.0008475780487060547 nb_pixel_total : 22247 time to create 1 rle with old method : 0.026308059692382812 length of segment : 162 time for calcul the mask position with numpy : 0.0015196800231933594 nb_pixel_total : 26624 time to create 1 rle with old method : 0.03076314926147461 length of segment : 379 time for calcul the mask position with numpy : 0.0017714500427246094 nb_pixel_total : 31180 time to create 1 rle with old method : 0.03612256050109863 length of segment : 230 time for calcul the mask position with numpy : 0.0012238025665283203 nb_pixel_total : 35634 time to create 1 rle with old method : 0.041158437728881836 length of segment : 276 time for calcul the mask position with numpy : 0.003895282745361328 nb_pixel_total : 79949 time to create 1 rle with old method : 0.17402887344360352 length of segment : 257 time for calcul the mask position with numpy : 0.0006089210510253906 nb_pixel_total : 10582 time to create 1 rle with old method : 0.014690399169921875 length of segment : 80 time for calcul the mask position with numpy : 0.0006351470947265625 nb_pixel_total : 13983 time to create 1 rle with old method : 0.01833510398864746 length of segment : 103 time for calcul the mask position with numpy : 0.000827789306640625 nb_pixel_total : 19548 time to create 1 rle with old method : 0.022915363311767578 length of segment : 167 time for calcul the mask position with numpy : 0.0004696846008300781 nb_pixel_total : 10660 time to create 1 rle with old method : 0.012701272964477539 length of segment : 79 time for calcul the mask position with numpy : 0.0037496089935302734 nb_pixel_total : 86149 time to create 1 rle with old method : 0.0965876579284668 length of segment : 307 time for calcul the mask position with numpy : 0.0007884502410888672 nb_pixel_total : 26407 time to create 1 rle with old method : 0.030287981033325195 length of segment : 214 time for calcul the mask position with numpy : 0.004818916320800781 nb_pixel_total : 153122 time to create 1 rle with new method : 0.007988214492797852 length of segment : 447 time for calcul the mask position with numpy : 0.00030231475830078125 nb_pixel_total : 6681 time to create 1 rle with old method : 0.007650852203369141 length of segment : 201 time for calcul the mask position with numpy : 0.0009183883666992188 nb_pixel_total : 31894 time to create 1 rle with old method : 0.036098480224609375 length of segment : 226 time for calcul the mask position with numpy : 0.0007429122924804688 nb_pixel_total : 17324 time to create 1 rle with old method : 0.020421504974365234 length of segment : 150 time for calcul the mask position with numpy : 0.0009098052978515625 nb_pixel_total : 18973 time to create 1 rle with old method : 0.022034883499145508 length of segment : 529 time for calcul the mask position with numpy : 0.0016150474548339844 nb_pixel_total : 54055 time to create 1 rle with old method : 0.061473846435546875 length of segment : 262 time for calcul the mask position with numpy : 0.0011639595031738281 nb_pixel_total : 34304 time to create 1 rle with old method : 0.040819406509399414 length of segment : 227 time for calcul the mask position with numpy : 0.002147674560546875 nb_pixel_total : 72666 time to create 1 rle with old method : 0.10650396347045898 length of segment : 318 time spent for convertir_results : 9.786359786987305 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 92 chid ids of type : 3594 Number RLEs to save : 25375 save missing photos in datou_result : time spend for datou_step_exec : 58.6838915348053 time spend to save output : 1.5101292133331299 total time spend for step 1 : 60.19402074813843 step2:crop_condition Thu May 15 13:01:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 6 ! batch 1 Loaded 92 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 61 About to insert : list_path_to_insert length 61 new photo from crops ! About to upload 61 photos upload in portfolio : 3736932 init cache_photo without model_param we have 61 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306903_3338443 we have uploaded 61 photos in the portfolio 3736932 time of upload the photos Elapsed time : 15.779444932937622 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 3736932 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306922_3338443 we have uploaded 11 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.9926953315734863 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306927_3338443 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.49700260162353516 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306951_3338443 we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.410336494445801 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306957_3338443 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7267091274261475 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306960_3338443 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7196059226989746 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747306962_3338443 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.1250216960906982 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1358431978, 1358431974, 1358431970, 1358431965, 1358431962, 1358431960] Looping around the photos to save general results len do output : 92 /1358497511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497574Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497576Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1358497777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431978', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431974', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431970', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431965', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431962', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431960', None, None, None, None, None, '2860281') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 282 time used for this insertion : 0.024588823318481445 save_final save missing photos in datou_result : time spend for datou_step_exec : 72.2336494922638 time spend to save output : 0.026633501052856445 total time spend for step 2 : 72.26028299331665 step3:rle_unique_nms_with_priority Thu May 15 13:02:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 92 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 2.7737035751342773 time for calcul the mask position with numpy : 0.3387181758880615 nb_pixel_total : 3944406 time to create 1 rle with new method : 0.980954647064209 time for calcul the mask position with numpy : 0.02657318115234375 nb_pixel_total : 49683 time to create 1 rle with old method : 0.05507636070251465 time for calcul the mask position with numpy : 0.2972142696380615 nb_pixel_total : 4237224 time to create 1 rle with new method : 0.6934843063354492 time for calcul the mask position with numpy : 0.026053190231323242 nb_pixel_total : 38279 time to create 1 rle with old method : 0.046063899993896484 time for calcul the mask position with numpy : 0.02972579002380371 nb_pixel_total : 24808 time to create 1 rle with old method : 0.027773618698120117 create new chi : 2.6004343032836914 time to delete rle : 0.0173795223236084 batch 1 Loaded 11 chid ids of type : 3594 ++++++Number RLEs to save : 6996 TO DO : save crop sub photo not yet done ! save time : 0.461641788482666 nb_obj : 15 nb_hashtags : 5 time to prepare the origin masks : 8.617292642593384 time for calcul the mask position with numpy : 0.7352275848388672 nb_pixel_total : 7452965 time to create 1 rle with new method : 1.099637746810913 time for calcul the mask position with numpy : 0.024733781814575195 nb_pixel_total : 25448 time to create 1 rle with old method : 0.0283355712890625 time for calcul the mask position with numpy : 0.025112390518188477 nb_pixel_total : 133960 time to create 1 rle with old method : 0.1553952693939209 time for calcul the mask position with numpy : 0.027996301651000977 nb_pixel_total : 17570 time to create 1 rle with old method : 0.02861618995666504 time for calcul the mask position with numpy : 0.02526712417602539 nb_pixel_total : 57918 time to create 1 rle with old method : 0.06440377235412598 time for calcul the mask position with numpy : 0.024931907653808594 nb_pixel_total : 19777 time to create 1 rle with old method : 0.021988868713378906 time for calcul the mask position with numpy : 0.02518916130065918 nb_pixel_total : 7501 time to create 1 rle with old method : 0.008137226104736328 time for calcul the mask position with numpy : 0.025584697723388672 nb_pixel_total : 122243 time to create 1 rle with old method : 0.1401522159576416 time for calcul the mask position with numpy : 0.03272414207458496 nb_pixel_total : 20482 time to create 1 rle with old method : 0.023311376571655273 time for calcul the mask position with numpy : 0.027890443801879883 nb_pixel_total : 180090 time to create 1 rle with new method : 0.7551462650299072 time for calcul the mask position with numpy : 0.027621984481811523 nb_pixel_total : 22439 time to create 1 rle with old method : 0.03573727607727051 time for calcul the mask position with numpy : 0.027002334594726562 nb_pixel_total : 14939 time to create 1 rle with old method : 0.01674675941467285 time for calcul the mask position with numpy : 0.026131153106689453 nb_pixel_total : 52383 time to create 1 rle with old method : 0.05992770195007324 time for calcul the mask position with numpy : 0.02612614631652832 nb_pixel_total : 45550 time to create 1 rle with old method : 0.05309152603149414 time for calcul the mask position with numpy : 0.027195453643798828 nb_pixel_total : 112557 time to create 1 rle with old method : 0.12333846092224121 time for calcul the mask position with numpy : 0.03166770935058594 nb_pixel_total : 8578 time to create 1 rle with old method : 0.009572505950927734 create new chi : 3.8367679119110107 time to delete rle : 0.0014171600341796875 batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 10565 TO DO : save crop sub photo not yet done ! save time : 0.6154835224151611 nb_obj : 24 nb_hashtags : 5 time to prepare the origin masks : 12.446730136871338 time for calcul the mask position with numpy : 0.8796181678771973 nb_pixel_total : 7380428 time to create 1 rle with new method : 1.525702714920044 time for calcul the mask position with numpy : 0.025957584381103516 nb_pixel_total : 6189 time to create 1 rle with old method : 0.007056474685668945 time for calcul the mask position with numpy : 0.032295942306518555 nb_pixel_total : 20895 time to create 1 rle with old method : 0.041478872299194336 time for calcul the mask position with numpy : 0.02937006950378418 nb_pixel_total : 25632 time to create 1 rle with old method : 0.028926372528076172 time for calcul the mask position with numpy : 0.02461981773376465 nb_pixel_total : 18605 time to create 1 rle with old method : 0.020786046981811523 time for calcul the mask position with numpy : 0.024754762649536133 nb_pixel_total : 28042 time to create 1 rle with old method : 0.031106233596801758 time for calcul the mask position with numpy : 0.02498483657836914 nb_pixel_total : 79083 time to create 1 rle with old method : 0.08719539642333984 time for calcul the mask position with numpy : 0.024543285369873047 nb_pixel_total : 33473 time to create 1 rle with old method : 0.04133152961730957 time for calcul the mask position with numpy : 0.03999733924865723 nb_pixel_total : 24838 time to create 1 rle with old method : 0.027563810348510742 time for calcul the mask position with numpy : 0.03969407081604004 nb_pixel_total : 18844 time to create 1 rle with old method : 0.020657062530517578 time for calcul the mask position with numpy : 0.039934396743774414 nb_pixel_total : 8969 time to create 1 rle with old method : 0.010074853897094727 time for calcul the mask position with numpy : 0.0406038761138916 nb_pixel_total : 8307 time to create 1 rle with old method : 0.00932002067565918 time for calcul the mask position with numpy : 0.03998756408691406 nb_pixel_total : 50312 time to create 1 rle with old method : 0.0556337833404541 time for calcul the mask position with numpy : 0.04388308525085449 nb_pixel_total : 74991 time to create 1 rle with old method : 0.08220362663269043 time for calcul the mask position with numpy : 0.04010939598083496 nb_pixel_total : 65331 time to create 1 rle with old method : 0.07532644271850586 time for calcul the mask position with numpy : 0.03977489471435547 nb_pixel_total : 14851 time to create 1 rle with old method : 0.016709089279174805 time for calcul the mask position with numpy : 0.025928974151611328 nb_pixel_total : 12268 time to create 1 rle with old method : 0.013849735260009766 time for calcul the mask position with numpy : 0.02451801300048828 nb_pixel_total : 52160 time to create 1 rle with old method : 0.057389259338378906 time for calcul the mask position with numpy : 0.02682352066040039 nb_pixel_total : 33896 time to create 1 rle with old method : 0.03964090347290039 time for calcul the mask position with numpy : 0.025742292404174805 nb_pixel_total : 18390 time to create 1 rle with old method : 0.020776033401489258 time for calcul the mask position with numpy : 0.02958536148071289 nb_pixel_total : 148445 time to create 1 rle with old method : 0.1628570556640625 time for calcul the mask position with numpy : 0.02601027488708496 nb_pixel_total : 89010 time to create 1 rle with old method : 0.09930419921875 time for calcul the mask position with numpy : 0.025049686431884766 nb_pixel_total : 10737 time to create 1 rle with old method : 0.01193857192993164 time for calcul the mask position with numpy : 0.02561640739440918 nb_pixel_total : 43058 time to create 1 rle with old method : 0.0472259521484375 time for calcul the mask position with numpy : 0.025036096572875977 nb_pixel_total : 27646 time to create 1 rle with old method : 0.030930280685424805 create new chi : 4.234134197235107 time to delete rle : 0.0016522407531738281 batch 1 Loaded 49 chid ids of type : 3594 ++++++++++++++++++++++++++Number RLEs to save : 13971 TO DO : save crop sub photo not yet done ! save time : 0.8113033771514893 nb_obj : 10 nb_hashtags : 4 time to prepare the origin masks : 6.03342866897583 time for calcul the mask position with numpy : 0.40988898277282715 nb_pixel_total : 8013740 time to create 1 rle with new method : 1.134782075881958 time for calcul the mask position with numpy : 0.04153871536254883 nb_pixel_total : 31369 time to create 1 rle with old method : 0.03449869155883789 time for calcul the mask position with numpy : 0.04180121421813965 nb_pixel_total : 52974 time to create 1 rle with old method : 0.058547019958496094 time for calcul the mask position with numpy : 0.04153299331665039 nb_pixel_total : 14012 time to create 1 rle with old method : 0.015986919403076172 time for calcul the mask position with numpy : 0.042632102966308594 nb_pixel_total : 16381 time to create 1 rle with old method : 0.018415212631225586 time for calcul the mask position with numpy : 0.04101419448852539 nb_pixel_total : 25447 time to create 1 rle with old method : 0.028774261474609375 time for calcul the mask position with numpy : 0.03424477577209473 nb_pixel_total : 34253 time to create 1 rle with old method : 0.03840827941894531 time for calcul the mask position with numpy : 0.02540874481201172 nb_pixel_total : 22242 time to create 1 rle with old method : 0.02765178680419922 time for calcul the mask position with numpy : 0.026848554611206055 nb_pixel_total : 41214 time to create 1 rle with old method : 0.04602813720703125 time for calcul the mask position with numpy : 0.028829097747802734 nb_pixel_total : 13149 time to create 1 rle with old method : 0.014900922775268555 time for calcul the mask position with numpy : 0.03114914894104004 nb_pixel_total : 29619 time to create 1 rle with old method : 0.03316760063171387 create new chi : 2.263315200805664 time to delete rle : 0.0015587806701660156 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++Number RLEs to save : 6314 TO DO : save crop sub photo not yet done ! save time : 0.4101884365081787 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 6.558274269104004 time for calcul the mask position with numpy : 0.5931301116943359 nb_pixel_total : 7505601 time to create 1 rle with new method : 0.8207988739013672 time for calcul the mask position with numpy : 0.026025056838989258 nb_pixel_total : 150631 time to create 1 rle with new method : 0.810600996017456 time for calcul the mask position with numpy : 0.03177142143249512 nb_pixel_total : 2275 time to create 1 rle with old method : 0.002686738967895508 time for calcul the mask position with numpy : 0.04066181182861328 nb_pixel_total : 59691 time to create 1 rle with old method : 0.06817770004272461 time for calcul the mask position with numpy : 0.05063676834106445 nb_pixel_total : 32317 time to create 1 rle with old method : 0.05083632469177246 time for calcul the mask position with numpy : 0.042919158935546875 nb_pixel_total : 22282 time to create 1 rle with old method : 0.024446964263916016 time for calcul the mask position with numpy : 0.04024028778076172 nb_pixel_total : 37213 time to create 1 rle with old method : 0.04151511192321777 time for calcul the mask position with numpy : 0.039391517639160156 nb_pixel_total : 17894 time to create 1 rle with old method : 0.019759654998779297 time for calcul the mask position with numpy : 0.03880119323730469 nb_pixel_total : 54516 time to create 1 rle with old method : 0.06009268760681152 time for calcul the mask position with numpy : 0.03571009635925293 nb_pixel_total : 23818 time to create 1 rle with old method : 0.02632284164428711 time for calcul the mask position with numpy : 0.025954484939575195 nb_pixel_total : 46620 time to create 1 rle with old method : 0.057179927825927734 time for calcul the mask position with numpy : 0.024257421493530273 nb_pixel_total : 58817 time to create 1 rle with old method : 0.06410932540893555 time for calcul the mask position with numpy : 0.024466991424560547 nb_pixel_total : 58912 time to create 1 rle with old method : 0.06492733955383301 time for calcul the mask position with numpy : 0.02495551109313965 nb_pixel_total : 7334 time to create 1 rle with old method : 0.007931947708129883 time for calcul the mask position with numpy : 0.0248262882232666 nb_pixel_total : 71792 time to create 1 rle with old method : 0.08067655563354492 time for calcul the mask position with numpy : 0.024956941604614258 nb_pixel_total : 54288 time to create 1 rle with old method : 0.059842824935913086 time for calcul the mask position with numpy : 0.024381637573242188 nb_pixel_total : 15706 time to create 1 rle with old method : 0.016975879669189453 time for calcul the mask position with numpy : 0.0256350040435791 nb_pixel_total : 74693 time to create 1 rle with old method : 0.0810861587524414 create new chi : 3.573094129562378 time to delete rle : 0.0015988349914550781 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 12533 TO DO : save crop sub photo not yet done ! save time : 0.7404143810272217 nb_obj : 20 nb_hashtags : 2 time to prepare the origin masks : 7.958805561065674 time for calcul the mask position with numpy : 0.7658584117889404 nb_pixel_total : 7525197 time to create 1 rle with new method : 1.1964621543884277 time for calcul the mask position with numpy : 0.026606321334838867 nb_pixel_total : 72666 time to create 1 rle with old method : 0.08011674880981445 time for calcul the mask position with numpy : 0.025243043899536133 nb_pixel_total : 34304 time to create 1 rle with old method : 0.03809809684753418 time for calcul the mask position with numpy : 0.027478933334350586 nb_pixel_total : 54055 time to create 1 rle with old method : 0.059835195541381836 time for calcul the mask position with numpy : 0.02461075782775879 nb_pixel_total : 10348 time to create 1 rle with old method : 0.011807918548583984 time for calcul the mask position with numpy : 0.023754119873046875 nb_pixel_total : 17324 time to create 1 rle with old method : 0.019218921661376953 time for calcul the mask position with numpy : 0.02530527114868164 nb_pixel_total : 31894 time to create 1 rle with old method : 0.036667823791503906 time for calcul the mask position with numpy : 0.02526998519897461 nb_pixel_total : 6681 time to create 1 rle with old method : 0.007491588592529297 time for calcul the mask position with numpy : 0.03114008903503418 nb_pixel_total : 153122 time to create 1 rle with new method : 0.7797255516052246 time for calcul the mask position with numpy : 0.02581620216369629 nb_pixel_total : 26407 time to create 1 rle with old method : 0.0292816162109375 time for calcul the mask position with numpy : 0.026302814483642578 nb_pixel_total : 86149 time to create 1 rle with old method : 0.09643316268920898 time for calcul the mask position with numpy : 0.02631664276123047 nb_pixel_total : 10660 time to create 1 rle with old method : 0.017028093338012695 time for calcul the mask position with numpy : 0.03723430633544922 nb_pixel_total : 19548 time to create 1 rle with old method : 0.023724794387817383 time for calcul the mask position with numpy : 0.03958272933959961 nb_pixel_total : 13983 time to create 1 rle with old method : 0.015598297119140625 time for calcul the mask position with numpy : 0.03915882110595703 nb_pixel_total : 10582 time to create 1 rle with old method : 0.012018203735351562 time for calcul the mask position with numpy : 0.04053473472595215 nb_pixel_total : 79949 time to create 1 rle with old method : 0.08875703811645508 time for calcul the mask position with numpy : 0.03610420227050781 nb_pixel_total : 35634 time to create 1 rle with old method : 0.05377626419067383 time for calcul the mask position with numpy : 0.027495622634887695 nb_pixel_total : 31180 time to create 1 rle with old method : 0.03548288345336914 time for calcul the mask position with numpy : 0.02527451515197754 nb_pixel_total : 26624 time to create 1 rle with old method : 0.029970645904541016 time for calcul the mask position with numpy : 0.026457548141479492 nb_pixel_total : 22247 time to create 1 rle with old method : 0.02475142478942871 time for calcul the mask position with numpy : 0.02688431739807129 nb_pixel_total : 25846 time to create 1 rle with old method : 0.028792142868041992 create new chi : 4.112483263015747 time to delete rle : 0.0015916824340820312 batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 11533 TO DO : save crop sub photo not yet done ! save time : 0.7146360874176025 map_output_result : {1358431978: (0.0, 'Should be the crop_list due to order', 0), 1358431974: (0.0, 'Should be the crop_list due to order', 0), 1358431970: (0.0, 'Should be the crop_list due to order', 0), 1358431965: (0.0, 'Should be the crop_list due to order', 0), 1358431962: (0.0, 'Should be the crop_list due to order', 0), 1358431960: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1358431978, 1358431974, 1358431970, 1358431965, 1358431962, 1358431960] Looping around the photos to save general results len do output : 6 /1358431978.Didn't retrieve data . /1358431974.Didn't retrieve data . /1358431970.Didn't retrieve data . /1358431965.Didn't retrieve data . /1358431962.Didn't retrieve data . /1358431960.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431978', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431974', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431970', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431965', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431962', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431960', None, None, None, None, None, '2860281') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.018965482711791992 save_final save missing photos in datou_result : time spend for datou_step_exec : 69.9262363910675 time spend to save output : 0.019358396530151367 total time spend for step 3 : 69.94559478759766 step4:ventilate_hashtags_in_portfolio Thu May 15 13:03:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23003831 get user id for portfolio 23003831 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23003831 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','carton','mal_croppe','environnement','metal','papier','flou','background','pehd','autre','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23003831 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','carton','mal_croppe','environnement','metal','papier','flou','background','pehd','autre','pet_fonce')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23003831 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','carton','mal_croppe','environnement','metal','papier','flou','background','pehd','autre','pet_fonce')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23007017,23007018,23007019,23007020,23007021,23007022,23007023,23007024,23007025,23007026,23007027?tags=pet_clair,carton,mal_croppe,environnement,metal,papier,flou,background,pehd,autre,pet_fonce Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1358431978, 1358431974, 1358431970, 1358431965, 1358431962, 1358431960] Looping around the photos to save general results len do output : 1 /23003831. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431978', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431974', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431970', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431965', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431962', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431960', None, None, None, None, None, '2860281') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.013988018035888672 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6790575981140137 time spend to save output : 0.014231204986572266 total time spend for step 4 : 1.693288803100586 step5:final Thu May 15 13:03:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1358431978: ('0.15962703751929014',), 1358431974: ('0.15962703751929014',), 1358431970: ('0.15962703751929014',), 1358431965: ('0.15962703751929014',), 1358431962: ('0.15962703751929014',), 1358431960: ('0.15962703751929014',)} new output for save of step final : {1358431978: ('0.15962703751929014',), 1358431974: ('0.15962703751929014',), 1358431970: ('0.15962703751929014',), 1358431965: ('0.15962703751929014',), 1358431962: ('0.15962703751929014',), 1358431960: ('0.15962703751929014',)} [1358431978, 1358431974, 1358431970, 1358431965, 1358431962, 1358431960] Looping around the photos to save general results len do output : 6 /1358431978.Didn't retrieve data . /1358431974.Didn't retrieve data . /1358431970.Didn't retrieve data . /1358431965.Didn't retrieve data . /1358431962.Didn't retrieve data . /1358431960.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431978', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431974', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431970', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431965', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431962', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431960', None, None, None, None, None, '2860281') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.018093109130859375 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11577153205871582 time spend to save output : 0.018452882766723633 total time spend for step 5 : 0.13422441482543945 step6:blur_detection Thu May 15 13:03:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5.jpg resize: (2160, 3840) 1358431978 -3.1394533065011245 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e.jpg resize: (2160, 3840) 1358431974 -6.419381497602298 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172.jpg resize: (2160, 3840) 1358431970 -6.6568312946230765 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f.jpg resize: (2160, 3840) 1358431965 -6.70411775272251 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b.jpg resize: (2160, 3840) 1358431962 -6.617744868244845 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb.jpg resize: (2160, 3840) 1358431960 -6.518073870388887 treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937812_0.png resize: (194, 337) 1358497511 -3.2096936558355913 treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937813_0.png resize: (98, 106) 1358497512 -3.806584226725782 treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937809_0.png resize: (220, 268) 1358497513 -4.28348526480556 treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937808_0.png resize: (259, 186) 1358497515 -3.825690384327808 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937820_0.png resize: (474, 498) 1358497516 -4.207933928192684 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937823_0.png resize: (94, 99) 1358497517 -1.4595947581692934 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937815_0.png resize: (344, 496) 1358497519 -3.7946250133779382 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937825_0.png resize: (324, 236) 1358497520 -2.7201954108159168 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937845_0.png resize: (344, 150) 1358497521 -2.851971747564318 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937852_0.png resize: (159, 150) 1358497523 -3.170977370577185 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937840_0.png resize: (261, 427) 1358497524 -4.237173817987283 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937838_0.png resize: (172, 122) 1358497525 -3.0028498152417527 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937837_0.png resize: (198, 127) 1358497526 -2.1664073976266707 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937836_0.png resize: (316, 226) 1358497528 -5.160908135331967 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937844_0.png resize: (159, 149) 1358497529 -2.9697751652375133 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937848_0.png resize: (172, 220) 1358497530 -2.716263368081637 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937830_0.png resize: (311, 183) 1358497532 -3.2635602725895225 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937849_0.png resize: (171, 210) 1358497533 -2.8340096096656824 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937832_0.png resize: (391, 344) 1358497535 -5.188247385359579 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937829_0.png resize: (186, 238) 1358497537 -3.8376829513325386 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937831_0.png resize: (105, 151) 1358497538 -3.255334498860755 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937834_0.png resize: (156, 161) 1358497539 -2.5125160806690383 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937842_0.png resize: (78, 153) 1358497541 -3.8202701200794613 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937839_0.png resize: (449, 209) 1358497543 -2.9039592963075287 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937843_0.png resize: (90, 120) 1358497544 -2.3718937255773413 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937858_0.png resize: (238, 173) 1358497545 -2.890841960558837 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937855_0.png resize: (220, 254) 1358497547 -3.7528042243569284 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937862_0.png resize: (184, 288) 1358497548 -1.6157379811863382 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937853_0.png resize: (195, 216) 1358497549 -4.053190131852562 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937854_0.png resize: (162, 128) 1358497551 -0.6974272050746797 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937859_0.png resize: (92, 226) 1358497552 -2.6098250037834494 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937860_0.png resize: (159, 123) 1358497554 -1.1648607008188414 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937868_0.png resize: (185, 383) 1358497555 0.5186923739600259 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937871_0.png resize: (215, 156) 1358497556 -3.1306589341448716 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937869_0.png resize: (342, 237) 1358497558 -0.4798140881686349 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937874_0.png resize: (293, 219) 1358497559 -3.396149211193621 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937872_0.png resize: (285, 316) 1358497560 -3.6597749386586047 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937865_0.png resize: (399, 347) 1358497561 -4.782186957958679 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937864_0.png resize: (192, 146) 1358497563 -3.8284389948586965 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937870_0.png resize: (290, 260) 1358497564 -4.134015683555304 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937875_0.png resize: (191, 165) 1358497565 -4.010265104383386 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937873_0.png resize: (443, 207) 1358497567 -2.627727430447615 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937867_0.png resize: (110, 81) 1358497568 -3.0505406108568778 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937877_0.png resize: (377, 245) 1358497569 -4.317438154597185 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937898_0.png resize: (224, 237) 1358497571 -4.547043858750567 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937890_0.png resize: (296, 446) 1358497572 -2.561894704779815 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937899_0.png resize: (317, 389) 1358497573 -4.324048365479189 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937881_0.png resize: (159, 163) 1358497574 -1.3002850686441552 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937883_0.png resize: (230, 285) 1358497576 -3.398104214862879 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937889_0.png resize: (77, 195) 1358497577 -3.574252695999444 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937895_0.png resize: (145, 180) 1358497578 -1.9834149491435886 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937886_0.png resize: (70, 209) 1358497580 -3.330693750821279 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937894_0.png resize: (226, 172) 1358497581 -4.450698474601196 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937888_0.png resize: (167, 148) 1358497582 -3.2493658917043575 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937884_0.png resize: (275, 164) 1358497583 -3.2725543329319464 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937896_0.png resize: (291, 184) 1358497585 -2.93406173953308 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937882_0.png resize: (377, 103) 1358497586 -3.736512316631126 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937893_0.png resize: (127, 90) 1358497587 -3.4210293576109607 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937880_0.png resize: (198, 184) 1358497589 -3.691436419356623 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937891_0.png resize: (214, 178) 1358497591 -3.2640638009229104 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937887_0.png resize: (100, 177) 1358497592 -0.8582516136297924 treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937811_0.png resize: (242, 271) 1358497635 -4.925209410222315 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937814_0.png resize: (110, 114) 1358497636 -4.206420283145601 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937821_0.png resize: (105, 248) 1358497637 -3.6787315628004 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937828_0.png resize: (282, 117) 1358497639 -2.5599762898051703 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937851_0.png resize: (181, 143) 1358497640 -3.485493525762885 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937835_0.png resize: (281, 188) 1358497641 -3.790482878729565 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937856_0.png resize: (195, 164) 1358497643 -3.0523715901106665 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937876_0.png resize: (126, 316) 1358497644 -3.9415822754478125 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937878_0.png resize: (293, 226) 1358497645 -3.3822402035029473 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937863_0.png resize: (258, 365) 1358497646 -2.865809473193229 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937879_0.png resize: (522, 457) 1358497648 -2.9594606272361 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937850_0.png resize: (193, 176) 1358497662 -3.7885043132361766 treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937810_0.png resize: (1616, 2920) 1358497755 -3.397300490262364 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937827_0.png resize: (615, 477) 1358497756 -2.5334434804879686 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937822_0.png resize: (408, 446) 1358497757 -4.733370340499136 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937817_0.png resize: (191, 417) 1358497758 -4.280772324139013 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937818_0.png resize: (182, 124) 1358497759 -2.9539289212813347 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937826_0.png resize: (144, 154) 1358497760 -3.8016465107007007 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937816_0.png resize: (431, 139) 1358497761 -4.83847581742516 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937833_0.png resize: (381, 470) 1358497762 -4.243319989665415 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937841_0.png resize: (196, 327) 1358497763 -3.662833818204808 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937846_0.png resize: (200, 245) 1358497764 -4.544790059512788 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937857_0.png resize: (274, 163) 1358497765 -4.51217973837264 treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937866_0.png resize: (303, 320) 1358497766 -4.610466597757361 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937892_0.png resize: (443, 473) 1358497767 -4.972382784180458 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937885_0.png resize: (252, 432) 1358497768 -3.8705193459763882 treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937897_0.png resize: (233, 342) 1358497769 -4.6640576440151085 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937819_0.png resize: (145, 219) 1358497772 -2.0624048856998796 treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937861_0.png resize: (289, 279) 1358497773 -2.816142346483323 treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937847_0.png resize: (355, 334) 1358497774 -3.5390916840711535 treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937824_0.png resize: (156, 164) 1358497777 -4.004328052200792 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 98 time used for this insertion : 0.020471811294555664 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 98 time used for this insertion : 0.022434711456298828 save missing photos in datou_result : time spend for datou_step_exec : 30.200135231018066 time spend to save output : 0.04827165603637695 total time spend for step 6 : 30.248406887054443 step7:brightness Thu May 15 13:04:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5.jpg treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e.jpg treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172.jpg treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f.jpg treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b.jpg treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb.jpg treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937812_0.png treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937813_0.png treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937809_0.png treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937808_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937820_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937823_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937815_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937825_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937845_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937852_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937840_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937838_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937837_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937836_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937844_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937848_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937830_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937849_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937832_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937829_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937831_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937834_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937842_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937839_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937843_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937858_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937855_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937862_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937853_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937854_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937859_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937860_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937868_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937871_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937869_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937874_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937872_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937865_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937864_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937870_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937875_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937873_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937867_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937877_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937898_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937890_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937899_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937881_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937883_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937889_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937895_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937886_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937894_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937888_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937884_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937896_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937882_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937893_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937880_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937891_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937887_0.png treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937811_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937814_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937821_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937828_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937851_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937835_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937856_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937876_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937878_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937863_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937879_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937850_0.png treat image : temp/1747306829_3338443_1358431978_601d3f9fdcdcf7aff396ef165a7cf8d5_rle_crop_3798937810_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937827_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937822_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937817_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937818_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937826_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937816_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937833_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937841_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937846_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937857_0.png treat image : temp/1747306829_3338443_1358431962_08ebb0ac669badbf8c77e30dd6f9b97b_rle_crop_3798937866_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937892_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937885_0.png treat image : temp/1747306829_3338443_1358431960_56c5aa20ec96677986cb22370a1e3acb_rle_crop_3798937897_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937819_0.png treat image : temp/1747306829_3338443_1358431965_8a785de0f4f574707dfaa558c4a9393f_rle_crop_3798937861_0.png treat image : temp/1747306829_3338443_1358431970_021c72e9f3417dc2293aa157605bd172_rle_crop_3798937847_0.png treat image : temp/1747306829_3338443_1358431974_26769ee33ca92a41d41739b2d7b2671e_rle_crop_3798937824_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 98 time used for this insertion : 0.01816582679748535 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 98 time used for this insertion : 0.020574569702148438 save missing photos in datou_result : time spend for datou_step_exec : 7.693365573883057 time spend to save output : 0.04476141929626465 total time spend for step 7 : 7.738126993179321 step8:velours_tree Thu May 15 13:04:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.14118409156799316 time spend to save output : 4.482269287109375e-05 total time spend for step 8 : 0.14122891426086426 step9:send_mail_cod Thu May 15 13:04:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23003831_15-05-2025_13_04_32.pdf 23007017 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette230070171747307072 23007018 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette230070181747307074 23007019 imagette230070191747307075 23007021 change filename to text .imagette230070211747307075 23007022 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette230070221747307075 23007023 imagette230070231747307076 23007024 imagette230070241747307076 23007025 change filename to text .imagette230070251747307076 23007026 change filename to text .change filename to text .imagette230070261747307077 23007027 change filename to text .imagette230070271747307077 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23003831 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23007017,23007018,23007019,23007020,23007021,23007022,23007023,23007024,23007025,23007026,23007027?tags=pet_clair,carton,mal_croppe,environnement,metal,papier,flou,background,pehd,autre,pet_fonce args[1358431978] : ((1358431978, -3.1394533065011245, 492609224), (1358431978, -1.123876583935473, 501862349), '0.15962703751929014') We are sending mail with results at report@fotonower.com args[1358431974] : ((1358431974, -6.419381497602298, 492609224), (1358431974, 0.3590674592065767, 2107752395), '0.15962703751929014') We are sending mail with results at report@fotonower.com args[1358431970] : ((1358431970, -6.6568312946230765, 492609224), (1358431970, 0.5174041300107644, 2107752395), '0.15962703751929014') We are sending mail with results at report@fotonower.com args[1358431965] : ((1358431965, -6.70411775272251, 492609224), (1358431965, 0.4465601708286203, 2107752395), '0.15962703751929014') We are sending mail with results at report@fotonower.com args[1358431962] : ((1358431962, -6.617744868244845, 492609224), (1358431962, 0.4059383161279088, 2107752395), '0.15962703751929014') We are sending mail with results at report@fotonower.com args[1358431960] : ((1358431960, -6.518073870388887, 492609224), (1358431960, 0.4665190167539184, 2107752395), '0.15962703751929014') We are sending mail with results at report@fotonower.com refus_total : 0.15962703751929014 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23003831 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23003831_15-05-2025_13_04_32.pdf results_Auto_P23003831_15-05-2025_13_04_32.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23003831_15-05-2025_13_04_32.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23003831','results_Auto_P23003831_15-05-2025_13_04_32.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23003831_15-05-2025_13_04_32.pdf','pdf','','0.99','0.15962703751929014') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23003831

https://www.fotonower.com/image?json=false&list_photos_id=1358431978
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1358431974
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1358431970
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1358431965
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1358431962
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1358431960
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 15.96%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/23007017?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23007018?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23007021?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23007022?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/23007025?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23007026?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23007027?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23003831_15-05-2025_13_04_32.pdf.

Lien vers velours :https://www.fotonower.com/velours/23007017,23007018,23007019,23007020,23007021,23007022,23007023,23007024,23007025,23007026,23007027?tags=pet_clair,carton,mal_croppe,environnement,metal,papier,flou,background,pehd,autre,pet_fonce.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 15 May 2025 11:04:39 GMT Content-Length: 0 Connection: close X-Message-Id: hnd50LFfRyykiPU23oe8Ow Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1358431978, 1358431974, 1358431970, 1358431965, 1358431962, 1358431960] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431978', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431974', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431970', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431965', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431962', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431960', None, None, None, None, None, '2860281') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.017161846160888672 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.7556586265563965 time spend to save output : 0.017323970794677734 total time spend for step 9 : 6.772982597351074 step10:split_time_score Thu May 15 13:04:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('06', 6),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 15052025 23003831 Nombre de photos uploadées : 6 / 23040 (0%) 15052025 23003831 Nombre de photos taguées (types de déchets): 0 / 6 (0%) 15052025 23003831 Nombre de photos taguées (volume) : 0 / 6 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ?????? elapsed_time : fill_and_build_computed_from_old_data 0.0004143714904785156 elapsed_time : insert_dashboard_record_day_entry 0.029538869857788086 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23003826 order by id desc limit 1 Qualite : 0.15962703751929014 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23003831_15-05-2025_13_04_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23003831 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23003831 AND mptpi.`type`=3594 To do Qualite : 0.14424263760288064 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23003838_15-05-2025_11_52_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23003838 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23003838 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23006130 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'15052025': {'nb_upload': 6, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1358431978, 1358431974, 1358431970, 1358431965, 1358431962, 1358431960] Looping around the photos to save general results len do output : 1 /23003831Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431978', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431974', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431970', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431965', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431962', None, None, None, None, None, '2860281') ('3318', None, None, None, None, None, None, None, '2860281') ('3318', '23003831', '1358431960', None, None, None, None, None, '2860281') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.014655113220214844 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.1005473136901855 time spend to save output : 0.014860868453979492 total time spend for step 10 : 2.115408182144165 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 6 set_done_treatment 115.02user 102.37system 4:16.00elapsed 84%CPU (0avgtext+0avgdata 4670728maxresident)k 964064inputs+134016outputs (3075major+10611530minor)pagefaults 0swaps