python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1860932 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2919258'] with mtr_portfolio_ids : ['23198775'] and first list_photo_ids : [] new path : /proc/1860932/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 0.8646044731140137 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed May 21 09:10:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-21 09:10:32.428102: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-21 09:10:32.442619: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-21 09:10:32.444517: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f1ba8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-21 09:10:32.444577: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-21 09:10:32.448534: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-21 09:10:32.682082: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x15c95a20 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-21 09:10:32.682126: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-21 09:10:32.683257: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 09:10:32.683626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 09:10:32.685767: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 09:10:32.688015: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 09:10:32.688371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 09:10:32.703136: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 09:10:32.706532: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 09:10:32.738927: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 09:10:32.741139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 09:10:32.741233: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 09:10:32.742160: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 09:10:32.742177: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 09:10:32.742186: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 09:10:32.744182: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6263 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-21 09:10:33.106107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 09:10:33.106190: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 09:10:33.106208: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 09:10:33.106224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 09:10:33.106239: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 09:10:33.106254: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 09:10:33.106269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 09:10:33.106284: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 09:10:33.107382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 09:10:33.108322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 09:10:33.108351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 09:10:33.108367: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 09:10:33.108381: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 09:10:33.108395: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 09:10:33.108409: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 09:10:33.108423: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 09:10:33.108437: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 09:10:33.109465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 09:10:33.109492: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 09:10:33.109500: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 09:10:33.109508: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 09:10:33.110589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6263 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-21 09:10:44.895056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 09:10:45.146736: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 09:10:46.922415: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.923086: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.923705: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.924279: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.924864: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.62G (2817927680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.925480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.36G (2536134912 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.926057: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.12G (2282521344 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.926096: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.926905: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.926929: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.934385: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.934442: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.935042: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.935061: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.941817: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.941899: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.942503: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.942521: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.974385: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.974437: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.975035: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.975053: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.981018: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.981073: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:46.981668: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:46.981686: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 09:10:47.037286: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.038253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.048890: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.049860: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.166447: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.167094: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.173157: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.173768: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.253433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.254074: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.259808: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.260404: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.270760: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.271417: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.274636: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.275242: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.292090: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.292688: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.297089: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.297684: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.558850: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.564340: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.566528: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.568121: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.574004: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.574683: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.591033: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.591758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.592738: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.593378: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.614890: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.616093: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.617826: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.618888: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.624493: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.625164: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.630465: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.631177: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.647397: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.648331: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.653843: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.654867: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.655748: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.656771: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.658237: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.659080: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.670375: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.671287: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.672165: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.672982: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.673826: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.674684: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.675603: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.676521: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.689450: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.690329: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.697176: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.697987: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.753597: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.754227: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.754822: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.755414: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.768258: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.768884: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.791726: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.792808: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.793797: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.794756: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.799655: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.800532: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.801514: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.802419: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.804869: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.804904: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-21 09:10:47.814444: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.815279: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.825921: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.827053: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.828139: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.829195: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.838746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 09:10:47.840375: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 36 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 7 Detection mask done ! Trying to reset tf kernel 1861537 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1045 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1844 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00644230842590332 nb_pixel_total : 176931 time to create 1 rle with new method : 0.014686107635498047 length of segment : 620 time for calcul the mask position with numpy : 0.0015676021575927734 nb_pixel_total : 81711 time to create 1 rle with old method : 0.09128713607788086 length of segment : 596 time for calcul the mask position with numpy : 0.004872322082519531 nb_pixel_total : 186567 time to create 1 rle with new method : 0.011243581771850586 length of segment : 609 time for calcul the mask position with numpy : 0.0006361007690429688 nb_pixel_total : 28643 time to create 1 rle with old method : 0.03281712532043457 length of segment : 181 time for calcul the mask position with numpy : 0.003805398941040039 nb_pixel_total : 236937 time to create 1 rle with new method : 0.01569986343383789 length of segment : 1049 time for calcul the mask position with numpy : 0.00025177001953125 nb_pixel_total : 9144 time to create 1 rle with old method : 0.010621309280395508 length of segment : 111 time for calcul the mask position with numpy : 0.004117727279663086 nb_pixel_total : 252360 time to create 1 rle with new method : 0.016071557998657227 length of segment : 778 time for calcul the mask position with numpy : 0.00339508056640625 nb_pixel_total : 139491 time to create 1 rle with old method : 0.18442869186401367 length of segment : 378 time for calcul the mask position with numpy : 0.0016231536865234375 nb_pixel_total : 99405 time to create 1 rle with old method : 0.11446928977966309 length of segment : 370 time for calcul the mask position with numpy : 0.00427556037902832 nb_pixel_total : 164452 time to create 1 rle with new method : 0.0205230712890625 length of segment : 1034 time for calcul the mask position with numpy : 0.003771543502807617 nb_pixel_total : 117421 time to create 1 rle with old method : 0.1675090789794922 length of segment : 387 time for calcul the mask position with numpy : 0.004347085952758789 nb_pixel_total : 91458 time to create 1 rle with old method : 0.11314964294433594 length of segment : 397 time for calcul the mask position with numpy : 0.05981159210205078 nb_pixel_total : 734442 time to create 1 rle with new method : 0.08524465560913086 length of segment : 1784 time for calcul the mask position with numpy : 0.0024385452270507812 nb_pixel_total : 60955 time to create 1 rle with old method : 0.06893253326416016 length of segment : 378 time for calcul the mask position with numpy : 0.004885435104370117 nb_pixel_total : 100134 time to create 1 rle with old method : 0.14183783531188965 length of segment : 592 time for calcul the mask position with numpy : 0.002066373825073242 nb_pixel_total : 46187 time to create 1 rle with old method : 0.053449392318725586 length of segment : 222 time for calcul the mask position with numpy : 0.03210163116455078 nb_pixel_total : 529357 time to create 1 rle with new method : 0.29733777046203613 length of segment : 1169 time for calcul the mask position with numpy : 0.0040891170501708984 nb_pixel_total : 126383 time to create 1 rle with old method : 0.1394639015197754 length of segment : 733 time for calcul the mask position with numpy : 0.0031774044036865234 nb_pixel_total : 117087 time to create 1 rle with old method : 0.13132572174072266 length of segment : 391 time for calcul the mask position with numpy : 0.004925251007080078 nb_pixel_total : 58113 time to create 1 rle with old method : 0.0747518539428711 length of segment : 706 time for calcul the mask position with numpy : 0.013300657272338867 nb_pixel_total : 353272 time to create 1 rle with new method : 0.14859652519226074 length of segment : 703 time for calcul the mask position with numpy : 0.011430501937866211 nb_pixel_total : 491823 time to create 1 rle with new method : 0.1055455207824707 length of segment : 735 time for calcul the mask position with numpy : 0.004151344299316406 nb_pixel_total : 143054 time to create 1 rle with old method : 0.161970853805542 length of segment : 504 time for calcul the mask position with numpy : 0.0029218196868896484 nb_pixel_total : 156278 time to create 1 rle with new method : 0.008054256439208984 length of segment : 320 time for calcul the mask position with numpy : 0.0010364055633544922 nb_pixel_total : 20099 time to create 1 rle with old method : 0.0349574089050293 length of segment : 257 time for calcul the mask position with numpy : 0.003782987594604492 nb_pixel_total : 102354 time to create 1 rle with old method : 0.11550021171569824 length of segment : 463 time for calcul the mask position with numpy : 0.005291461944580078 nb_pixel_total : 173392 time to create 1 rle with new method : 0.019005775451660156 length of segment : 848 time for calcul the mask position with numpy : 0.0020401477813720703 nb_pixel_total : 76106 time to create 1 rle with old method : 0.09258008003234863 length of segment : 384 time for calcul the mask position with numpy : 0.0024421215057373047 nb_pixel_total : 93586 time to create 1 rle with old method : 0.10694599151611328 length of segment : 236 time for calcul the mask position with numpy : 0.0002493858337402344 nb_pixel_total : 11319 time to create 1 rle with old method : 0.013329744338989258 length of segment : 124 time for calcul the mask position with numpy : 0.011076927185058594 nb_pixel_total : 284899 time to create 1 rle with new method : 0.03883957862854004 length of segment : 1341 time for calcul the mask position with numpy : 0.007462501525878906 nb_pixel_total : 193492 time to create 1 rle with new method : 0.017454862594604492 length of segment : 594 time for calcul the mask position with numpy : 0.0018763542175292969 nb_pixel_total : 36448 time to create 1 rle with old method : 0.041959524154663086 length of segment : 407 time for calcul the mask position with numpy : 0.007475137710571289 nb_pixel_total : 236807 time to create 1 rle with new method : 0.012881040573120117 length of segment : 714 time for calcul the mask position with numpy : 0.004891395568847656 nb_pixel_total : 109060 time to create 1 rle with old method : 0.13587641716003418 length of segment : 1190 time for calcul the mask position with numpy : 0.0031282901763916016 nb_pixel_total : 65111 time to create 1 rle with old method : 0.0736548900604248 length of segment : 353 time for calcul the mask position with numpy : 0.0030143260955810547 nb_pixel_total : 90473 time to create 1 rle with old method : 0.11113429069519043 length of segment : 313 time for calcul the mask position with numpy : 0.0005340576171875 nb_pixel_total : 21721 time to create 1 rle with old method : 0.026015758514404297 length of segment : 247 time for calcul the mask position with numpy : 0.0011882781982421875 nb_pixel_total : 16414 time to create 1 rle with old method : 0.02082085609436035 length of segment : 296 time for calcul the mask position with numpy : 0.0005786418914794922 nb_pixel_total : 17789 time to create 1 rle with old method : 0.02059650421142578 length of segment : 188 time for calcul the mask position with numpy : 0.0008907318115234375 nb_pixel_total : 17343 time to create 1 rle with old method : 0.0207521915435791 length of segment : 132 time for calcul the mask position with numpy : 0.005687236785888672 nb_pixel_total : 161800 time to create 1 rle with new method : 0.011562108993530273 length of segment : 494 time for calcul the mask position with numpy : 0.0014142990112304688 nb_pixel_total : 36358 time to create 1 rle with old method : 0.04447507858276367 length of segment : 330 time for calcul the mask position with numpy : 0.004144906997680664 nb_pixel_total : 63513 time to create 1 rle with old method : 0.07179403305053711 length of segment : 470 time for calcul the mask position with numpy : 0.0004818439483642578 nb_pixel_total : 8031 time to create 1 rle with old method : 0.009568929672241211 length of segment : 105 time for calcul the mask position with numpy : 0.002816438674926758 nb_pixel_total : 106693 time to create 1 rle with old method : 0.12782716751098633 length of segment : 228 time for calcul the mask position with numpy : 0.0060062408447265625 nb_pixel_total : 164197 time to create 1 rle with new method : 0.015608072280883789 length of segment : 1122 time for calcul the mask position with numpy : 0.0009174346923828125 nb_pixel_total : 36307 time to create 1 rle with old method : 0.04329347610473633 length of segment : 129 time for calcul the mask position with numpy : 0.003919839859008789 nb_pixel_total : 171878 time to create 1 rle with new method : 0.007002115249633789 length of segment : 543 time for calcul the mask position with numpy : 0.0015270709991455078 nb_pixel_total : 66492 time to create 1 rle with old method : 0.0822443962097168 length of segment : 161 time for calcul the mask position with numpy : 0.005843162536621094 nb_pixel_total : 269262 time to create 1 rle with new method : 0.009785890579223633 length of segment : 645 time for calcul the mask position with numpy : 0.002273082733154297 nb_pixel_total : 111329 time to create 1 rle with old method : 0.12597894668579102 length of segment : 314 time for calcul the mask position with numpy : 0.011144399642944336 nb_pixel_total : 204625 time to create 1 rle with new method : 0.014127731323242188 length of segment : 885 time for calcul the mask position with numpy : 0.009387731552124023 nb_pixel_total : 296082 time to create 1 rle with new method : 0.012206077575683594 length of segment : 660 time for calcul the mask position with numpy : 0.0027153491973876953 nb_pixel_total : 117205 time to create 1 rle with old method : 0.15789175033569336 length of segment : 792 time for calcul the mask position with numpy : 0.0023658275604248047 nb_pixel_total : 96987 time to create 1 rle with old method : 0.11601662635803223 length of segment : 369 time for calcul the mask position with numpy : 0.0013356208801269531 nb_pixel_total : 15588 time to create 1 rle with old method : 0.01839423179626465 length of segment : 278 time for calcul the mask position with numpy : 0.0006957054138183594 nb_pixel_total : 31822 time to create 1 rle with old method : 0.03733563423156738 length of segment : 155 time for calcul the mask position with numpy : 0.0007395744323730469 nb_pixel_total : 32340 time to create 1 rle with old method : 0.037442922592163086 length of segment : 269 time for calcul the mask position with numpy : 0.004073381423950195 nb_pixel_total : 210955 time to create 1 rle with new method : 0.010038137435913086 length of segment : 816 time spent for convertir_results : 11.003241539001465 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 60 chid ids of type : 3594 Number RLEs to save : 31599 save missing photos in datou_result : time spend for datou_step_exec : 56.303555965423584 time spend to save output : 16.15122151374817 total time spend for step 1 : 72.45477747917175 step2:crop_condition Wed May 21 09:11:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 60 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 44 About to insert : list_path_to_insert length 44 new photo from crops ! About to upload 44 photos upload in portfolio : 3736932 init cache_photo without model_param we have 44 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747811528_1860932 we have uploaded 44 photos in the portfolio 3736932 time of upload the photos Elapsed time : 11.824152946472168 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 3736932 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747811548_1860932 we have uploaded 11 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.96639084815979 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747811554_1860932 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.0208053588867188 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747811558_1860932 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7870917320251465 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747811560_1860932 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6268496513366699 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359965362, 1359965279, 1359965262, 1359965258, 1359965254] Looping around the photos to save general results len do output : 60 /1359981098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981139Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359981255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965362', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965279', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965262', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965258', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965254', None, None, None, None, None, '2919258') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 185 time used for this insertion : 0.30083632469177246 save_final save missing photos in datou_result : time spend for datou_step_exec : 60.18759250640869 time spend to save output : 0.33565330505371094 total time spend for step 2 : 60.5232458114624 step3:rle_unique_nms_with_priority Wed May 21 09:12:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 60 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 10.623698949813843 time for calcul the mask position with numpy : 0.2787494659423828 nb_pixel_total : 5051779 time to create 1 rle with new method : 0.5010583400726318 time for calcul the mask position with numpy : 0.04593634605407715 nb_pixel_total : 58102 time to create 1 rle with old method : 0.06762170791625977 time for calcul the mask position with numpy : 0.04351353645324707 nb_pixel_total : 2541 time to create 1 rle with old method : 0.0033473968505859375 time for calcul the mask position with numpy : 0.04822731018066406 nb_pixel_total : 126383 time to create 1 rle with old method : 0.1401050090789795 time for calcul the mask position with numpy : 0.049982309341430664 nb_pixel_total : 529357 time to create 1 rle with new method : 0.5454580783843994 time for calcul the mask position with numpy : 0.05064535140991211 nb_pixel_total : 46187 time to create 1 rle with old method : 0.05129694938659668 time for calcul the mask position with numpy : 0.05132293701171875 nb_pixel_total : 100134 time to create 1 rle with old method : 0.11393523216247559 time for calcul the mask position with numpy : 0.04596209526062012 nb_pixel_total : 60955 time to create 1 rle with old method : 0.0705869197845459 time for calcul the mask position with numpy : 0.05732131004333496 nb_pixel_total : 734442 time to create 1 rle with new method : 0.5926802158355713 time for calcul the mask position with numpy : 0.04511404037475586 nb_pixel_total : 91458 time to create 1 rle with old method : 0.10934185981750488 time for calcul the mask position with numpy : 0.04622817039489746 nb_pixel_total : 117421 time to create 1 rle with old method : 0.12945008277893066 time for calcul the mask position with numpy : 0.05156850814819336 nb_pixel_total : 164452 time to create 1 rle with new method : 2.072735548019409 time for calcul the mask position with numpy : 0.041471242904663086 nb_pixel_total : 99405 time to create 1 rle with old method : 0.13628077507019043 time for calcul the mask position with numpy : 0.045020341873168945 nb_pixel_total : 139491 time to create 1 rle with old method : 0.16972994804382324 time for calcul the mask position with numpy : 0.0477445125579834 nb_pixel_total : 252360 time to create 1 rle with new method : 0.90267014503479 time for calcul the mask position with numpy : 0.04678797721862793 nb_pixel_total : 9144 time to create 1 rle with old method : 0.017253398895263672 time for calcul the mask position with numpy : 0.05317878723144531 nb_pixel_total : 236937 time to create 1 rle with new method : 0.9544038772583008 time for calcul the mask position with numpy : 0.042388916015625 nb_pixel_total : 28643 time to create 1 rle with old method : 0.03236651420593262 time for calcul the mask position with numpy : 0.042189598083496094 nb_pixel_total : 186567 time to create 1 rle with new method : 0.5582940578460693 time for calcul the mask position with numpy : 0.04175209999084473 nb_pixel_total : 81711 time to create 1 rle with old method : 0.09154653549194336 time for calcul the mask position with numpy : 0.042681217193603516 nb_pixel_total : 176931 time to create 1 rle with new method : 0.6662304401397705 create new chi : 9.393381357192993 time to delete rle : 0.029549121856689453 batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 26633 TO DO : save crop sub photo not yet done ! save time : 15.118362665176392 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 6.7708046436309814 time for calcul the mask position with numpy : 0.772228479385376 nb_pixel_total : 6416991 time to create 1 rle with new method : 0.968146562576294 time for calcul the mask position with numpy : 0.05255579948425293 nb_pixel_total : 284865 time to create 1 rle with new method : 0.5855109691619873 time for calcul the mask position with numpy : 0.04918932914733887 nb_pixel_total : 1976 time to create 1 rle with old method : 0.0039010047912597656 time for calcul the mask position with numpy : 0.052557945251464844 nb_pixel_total : 93586 time to create 1 rle with old method : 0.1180582046508789 time for calcul the mask position with numpy : 0.03499007225036621 nb_pixel_total : 76106 time to create 1 rle with old method : 0.08832383155822754 time for calcul the mask position with numpy : 0.027178049087524414 nb_pixel_total : 153996 time to create 1 rle with new method : 0.7971470355987549 time for calcul the mask position with numpy : 0.025054931640625 nb_pixel_total : 102354 time to create 1 rle with old method : 0.11339044570922852 time for calcul the mask position with numpy : 0.02691340446472168 nb_pixel_total : 20099 time to create 1 rle with old method : 0.0224759578704834 time for calcul the mask position with numpy : 0.028981685638427734 nb_pixel_total : 156278 time to create 1 rle with new method : 0.6856846809387207 time for calcul the mask position with numpy : 0.02608966827392578 nb_pixel_total : 143054 time to create 1 rle with old method : 0.19550561904907227 time for calcul the mask position with numpy : 0.0352630615234375 nb_pixel_total : 491823 time to create 1 rle with new method : 0.5051498413085938 time for calcul the mask position with numpy : 0.029109477996826172 nb_pixel_total : 353272 time to create 1 rle with new method : 0.9230473041534424 create new chi : 6.354799270629883 time to delete rle : 0.001829385757446289 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 13515 TO DO : save crop sub photo not yet done ! save time : 9.4964120388031 nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 4.878779172897339 time for calcul the mask position with numpy : 0.35776567459106445 nb_pixel_total : 6953774 time to create 1 rle with new method : 0.46483850479125977 time for calcul the mask position with numpy : 0.031476736068725586 nb_pixel_total : 159673 time to create 1 rle with new method : 1.6912074089050293 time for calcul the mask position with numpy : 0.02480316162109375 nb_pixel_total : 106593 time to create 1 rle with old method : 0.11714792251586914 time for calcul the mask position with numpy : 0.027934551239013672 nb_pixel_total : 8031 time to create 1 rle with old method : 0.013577461242675781 time for calcul the mask position with numpy : 0.02911210060119629 nb_pixel_total : 63513 time to create 1 rle with old method : 0.0740361213684082 time for calcul the mask position with numpy : 0.0369877815246582 nb_pixel_total : 36358 time to create 1 rle with old method : 0.04021143913269043 time for calcul the mask position with numpy : 0.041433095932006836 nb_pixel_total : 161800 time to create 1 rle with new method : 0.7890281677246094 time for calcul the mask position with numpy : 0.038840293884277344 nb_pixel_total : 17343 time to create 1 rle with old method : 0.019147396087646484 time for calcul the mask position with numpy : 0.03353428840637207 nb_pixel_total : 17789 time to create 1 rle with old method : 0.018993616104125977 time for calcul the mask position with numpy : 0.030704736709594727 nb_pixel_total : 16414 time to create 1 rle with old method : 0.017991304397583008 time for calcul the mask position with numpy : 0.032270193099975586 nb_pixel_total : 21721 time to create 1 rle with old method : 0.024229764938354492 time for calcul the mask position with numpy : 0.03426098823547363 nb_pixel_total : 90473 time to create 1 rle with old method : 0.09861278533935547 time for calcul the mask position with numpy : 0.025616168975830078 nb_pixel_total : 65111 time to create 1 rle with old method : 0.07108783721923828 time for calcul the mask position with numpy : 0.027202367782592773 nb_pixel_total : 109060 time to create 1 rle with old method : 0.12068700790405273 time for calcul the mask position with numpy : 0.025712013244628906 nb_pixel_total : 236807 time to create 1 rle with new method : 0.5187432765960693 time for calcul the mask position with numpy : 0.025671720504760742 nb_pixel_total : 36448 time to create 1 rle with old method : 0.043715715408325195 time for calcul the mask position with numpy : 0.02468109130859375 nb_pixel_total : 193492 time to create 1 rle with new method : 0.6462035179138184 create new chi : 6.111783981323242 time to delete rle : 0.002153635025024414 batch 1 Loaded 33 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 16085 TO DO : save crop sub photo not yet done ! save time : 9.46341872215271 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 4.820525407791138 time for calcul the mask position with numpy : 0.6578283309936523 nb_pixel_total : 7138425 time to create 1 rle with new method : 0.5261416435241699 time for calcul the mask position with numpy : 0.0276491641998291 nb_pixel_total : 296082 time to create 1 rle with new method : 0.6469759941101074 time for calcul the mask position with numpy : 0.030262470245361328 nb_pixel_total : 204625 time to create 1 rle with new method : 0.7256112098693848 time for calcul the mask position with numpy : 0.028507232666015625 nb_pixel_total : 111329 time to create 1 rle with old method : 0.12276768684387207 time for calcul the mask position with numpy : 0.029761552810668945 nb_pixel_total : 269262 time to create 1 rle with new method : 0.7732181549072266 time for calcul the mask position with numpy : 0.026000022888183594 nb_pixel_total : 66492 time to create 1 rle with old method : 0.08726215362548828 time for calcul the mask position with numpy : 0.02910304069519043 nb_pixel_total : 171878 time to create 1 rle with new method : 0.6446073055267334 time for calcul the mask position with numpy : 0.026816606521606445 nb_pixel_total : 36307 time to create 1 rle with old method : 0.040993690490722656 create new chi : 4.579225301742554 time to delete rle : 0.0013470649719238281 batch 1 Loaded 15 chid ids of type : 3594 +++++++Number RLEs to save : 8834 TO DO : save crop sub photo not yet done ! save time : 6.775566577911377 nb_obj : 6 nb_hashtags : 1 time to prepare the origin masks : 2.678039312362671 time for calcul the mask position with numpy : 0.3064298629760742 nb_pixel_total : 7789609 time to create 1 rle with new method : 0.7718989849090576 time for calcul the mask position with numpy : 0.02555108070373535 nb_pixel_total : 210849 time to create 1 rle with new method : 0.7626078128814697 time for calcul the mask position with numpy : 0.03701329231262207 nb_pixel_total : 32340 time to create 1 rle with old method : 0.03517794609069824 time for calcul the mask position with numpy : 0.03324270248413086 nb_pixel_total : 31822 time to create 1 rle with old method : 0.03527522087097168 time for calcul the mask position with numpy : 0.025503873825073242 nb_pixel_total : 15588 time to create 1 rle with old method : 0.01693582534790039 time for calcul the mask position with numpy : 0.025264501571655273 nb_pixel_total : 96987 time to create 1 rle with old method : 0.10739660263061523 time for calcul the mask position with numpy : 0.025195598602294922 nb_pixel_total : 117205 time to create 1 rle with old method : 0.12486600875854492 create new chi : 2.404846668243408 time to delete rle : 0.0011212825775146484 batch 1 Loaded 13 chid ids of type : 3594 +++++++++++Number RLEs to save : 7500 TO DO : save crop sub photo not yet done ! save time : 3.9204699993133545 map_output_result : {1359965362: (0.0, 'Should be the crop_list due to order', 0), 1359965279: (0.0, 'Should be the crop_list due to order', 0), 1359965262: (0.0, 'Should be the crop_list due to order', 0), 1359965258: (0.0, 'Should be the crop_list due to order', 0), 1359965254: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359965362, 1359965279, 1359965262, 1359965258, 1359965254] Looping around the photos to save general results len do output : 5 /1359965362.Didn't retrieve data . /1359965279.Didn't retrieve data . /1359965262.Didn't retrieve data . /1359965258.Didn't retrieve data . /1359965254.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965362', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965279', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965262', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965258', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965254', None, None, None, None, None, '2919258') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.06415796279907227 save_final save missing photos in datou_result : time spend for datou_step_exec : 104.59628129005432 time spend to save output : 0.06659150123596191 total time spend for step 3 : 104.66287279129028 step4:ventilate_hashtags_in_portfolio Wed May 21 09:14:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23198775 get user id for portfolio 23198775 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23198775 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','metal','papier','pet_clair','carton','background','autre','flou','pet_fonce','mal_croppe','pehd')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23198775 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','metal','papier','pet_clair','carton','background','autre','flou','pet_fonce','mal_croppe','pehd')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23198775 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','metal','papier','pet_clair','carton','background','autre','flou','pet_fonce','mal_croppe','pehd')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23199390,23199391,23199392,23199393,23199394,23199395,23199396,23199397,23199398,23199399,23199400?tags=environnement,metal,papier,pet_clair,carton,background,autre,flou,pet_fonce,mal_croppe,pehd Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359965362, 1359965279, 1359965262, 1359965258, 1359965254] Looping around the photos to save general results len do output : 1 /23198775. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965362', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965279', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965262', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965258', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965254', None, None, None, None, None, '2919258') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.08071374893188477 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.948730707168579 time spend to save output : 0.08107399940490723 total time spend for step 4 : 2.0298047065734863 step5:final Wed May 21 09:14:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359965362: ('0.1958290412808642',), 1359965279: ('0.1958290412808642',), 1359965262: ('0.1958290412808642',), 1359965258: ('0.1958290412808642',), 1359965254: ('0.1958290412808642',)} new output for save of step final : {1359965362: ('0.1958290412808642',), 1359965279: ('0.1958290412808642',), 1359965262: ('0.1958290412808642',), 1359965258: ('0.1958290412808642',), 1359965254: ('0.1958290412808642',)} [1359965362, 1359965279, 1359965262, 1359965258, 1359965254] Looping around the photos to save general results len do output : 5 /1359965362.Didn't retrieve data . /1359965279.Didn't retrieve data . /1359965262.Didn't retrieve data . /1359965258.Didn't retrieve data . /1359965254.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965362', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965279', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965262', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965258', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965254', None, None, None, None, None, '2919258') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.05371665954589844 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1690201759338379 time spend to save output : 0.05399203300476074 total time spend for step 5 : 0.22301220893859863 step6:blur_detection Wed May 21 09:14:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1.jpg resize: (2160, 3840) 1359965362 -5.7681982637572995 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7.jpg resize: (2160, 3840) 1359965279 -5.389897555491412 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884.jpg resize: (2160, 3840) 1359965262 -5.529678460617215 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9.jpg resize: (2160, 3840) 1359965258 -3.625401726055083 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5.jpg resize: (2160, 3840) 1359965254 -3.1406418875714457 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372286_0.png resize: (547, 308) 1359981098 -0.34679700157978577 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372276_0.png resize: (820, 441) 1359981099 -3.3140754242278603 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372279_0.png resize: (977, 1049) 1359981101 -3.0273358325634696 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372270_0.png resize: (157, 244) 1359981102 -3.7641077568899246 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372277_0.png resize: (381, 553) 1359981103 -3.3345499513843238 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372273_0.png resize: (774, 479) 1359981105 -3.7878357059322108 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372282_0.png resize: (222, 314) 1359981106 -3.758750212733015 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372272_0.png resize: (111, 100) 1359981107 -2.3391754032848207 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372284_0.png resize: (539, 318) 1359981109 -3.7102256746076585 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372267_0.png resize: (620, 454) 1359981110 -4.472122485722539 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372283_0.png resize: (1145, 928) 1359981112 -3.9270574526788993 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372271_0.png resize: (667, 514) 1359981113 -5.183959507550028 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372269_0.png resize: (482, 590) 1359981114 -2.2063606336572863 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372294_0.png resize: (384, 301) 1359981115 -3.469710864249631 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372287_0.png resize: (617, 749) 1359981117 -2.661165628673299 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372293_0.png resize: (502, 703) 1359981118 -4.295334705107993 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372290_0.png resize: (318, 662) 1359981119 -1.2574032940736872 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372289_0.png resize: (396, 635) 1359981121 -4.527160457040854 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372292_0.png resize: (324, 592) 1359981122 -4.536287598552917 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372296_0.png resize: (122, 148) 1359981123 -4.793175929057948 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372297_0.png resize: (1321, 322) 1359981125 -5.346132457994274 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372291_0.png resize: (257, 129) 1359981126 -3.53871215589855 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372295_0.png resize: (232, 601) 1359981127 -2.2743818922214953 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372309_0.png resize: (330, 145) 1359981129 -3.6304755410436926 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372302_0.png resize: (322, 392) 1359981130 -2.2238256754485715 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372310_0.png resize: (465, 396) 1359981131 -2.387761073621791 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372298_0.png resize: (574, 648) 1359981133 -2.3916004606599164 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372303_0.png resize: (312, 526) 1359981134 -3.7780306427671366 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372299_0.png resize: (376, 216) 1359981135 -3.891642549177775 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372300_0.png resize: (692, 545) 1359981137 -3.496780297882715 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372307_0.png resize: (131, 207) 1359981138 -3.4728479755478086 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372304_0.png resize: (209, 153) 1359981139 -4.471516871014774 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372305_0.png resize: (214, 168) 1359981141 -3.0361441758128724 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372306_0.png resize: (182, 128) 1359981142 -4.120769024514219 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372301_0.png resize: (797, 230) 1359981144 -4.552967856575986 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372311_0.png resize: (105, 132) 1359981145 -4.345634250309194 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372317_0.png resize: (645, 499) 1359981146 -4.90477657852532 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372319_0.png resize: (885, 414) 1359981148 -4.046974850056036 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372322_0.png resize: (369, 356) 1359981149 -4.562985729362514 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372326_0.png resize: (575, 774) 1359981151 -3.823424446486587 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372321_0.png resize: (574, 301) 1359981154 -2.596972903667784 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372323_0.png resize: (336, 216) 1359981155 -2.7127847345659766 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372324_0.png resize: (130, 379) 1359981157 -3.4462580123308144 treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372325_0.png resize: (269, 177) 1359981158 -1.6492674604067472 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372285_0.png resize: (386, 541) 1359981204 -3.335750082545986 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372275_0.png resize: (370, 340) 1359981205 -4.367623453360856 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372268_0.png resize: (553, 237) 1359981206 -4.226030153312998 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372280_0.png resize: (318, 305) 1359981207 -3.6959313105809932 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372278_0.png resize: (397, 364) 1359981208 -4.591568518164642 treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372288_0.png resize: (532, 1277) 1359981209 -4.469085658355247 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372313_0.png resize: (742, 410) 1359981210 -4.287090492936474 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372314_0.png resize: (115, 430) 1359981211 -3.603586199726659 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372315_0.png resize: (440, 691) 1359981212 -2.7504495275856695 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372318_0.png resize: (311, 511) 1359981213 -3.0498706649995633 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372316_0.png resize: (156, 572) 1359981214 -3.699869175366334 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372281_0.png resize: (532, 379) 1359981217 -3.2149127054766056 treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372274_0.png resize: (377, 504) 1359981219 -3.62992304476069 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372312_0.png resize: (206, 670) 1359981220 -4.796623484868798 treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372320_0.png resize: (653, 559) 1359981237 -3.0324051021422576 treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372308_0.png resize: (484, 457) 1359981255 -4.970081911281749 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 65 time used for this insertion : 0.018812894821166992 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 65 time used for this insertion : 0.38979625701904297 save missing photos in datou_result : time spend for datou_step_exec : 26.27814030647278 time spend to save output : 0.4133446216583252 total time spend for step 6 : 26.691484928131104 step7:brightness Wed May 21 09:14:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1.jpg treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7.jpg treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884.jpg treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9.jpg treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5.jpg treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372286_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372276_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372279_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372270_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372277_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372273_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372282_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372272_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372284_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372267_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372283_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372271_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372269_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372294_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372287_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372293_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372290_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372289_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372292_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372296_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372297_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372291_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372295_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372309_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372302_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372310_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372298_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372303_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372299_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372300_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372307_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372304_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372305_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372306_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372301_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372311_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372317_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372319_0.png treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372322_0.png treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372326_0.png treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372321_0.png treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372323_0.png treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372324_0.png treat image : temp/1747811427_1860932_1359965254_a2dfdf5d4951959a446b2b148872afb5_rle_crop_3807372325_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372285_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372275_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372268_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372280_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372278_0.png treat image : temp/1747811427_1860932_1359965279_22c945b1138faabffec54c9a6007eea7_rle_crop_3807372288_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372313_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372314_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372315_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372318_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372316_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372281_0.png treat image : temp/1747811427_1860932_1359965362_d8df60ac0f89032bff7ee3a89d3351e1_rle_crop_3807372274_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372312_0.png treat image : temp/1747811427_1860932_1359965258_5caa7363e5f7ad041cb711a0f78689e9_rle_crop_3807372320_0.png treat image : temp/1747811427_1860932_1359965262_740e74e94119bc620faab91b2675b884_rle_crop_3807372308_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 65 time used for this insertion : 0.017839908599853516 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 65 time used for this insertion : 0.02681708335876465 save missing photos in datou_result : time spend for datou_step_exec : 6.0892603397369385 time spend to save output : 0.049794673919677734 total time spend for step 7 : 6.139055013656616 step8:velours_tree Wed May 21 09:15:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.6996755599975586 time spend to save output : 0.00021982192993164062 total time spend for step 8 : 0.6998953819274902 step9:send_mail_cod Wed May 21 09:15:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23198775_21-05-2025_09_15_02.pdf 23199391 imagette231993911747811702 23199392 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231993921747811702 23199393 change filename to text .change filename to text .change filename to text .imagette231993931747811703 23199394 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231993941747811703 23199395 imagette231993951747811705 23199396 imagette231993961747811705 23199397 imagette231993971747811705 23199398 change filename to text .imagette231993981747811705 23199399 imagette231993991747811705 23199400 change filename to text .imagette231994001747811705 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23198775 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23199390,23199391,23199392,23199393,23199394,23199395,23199396,23199397,23199398,23199399,23199400?tags=environnement,metal,papier,pet_clair,carton,background,autre,flou,pet_fonce,mal_croppe,pehd args[1359965362] : ((1359965362, -5.7681982637572995, 492609224), (1359965362, 0.4824661562305835, 2107752395), '0.1958290412808642') We are sending mail with results at report@fotonower.com args[1359965279] : ((1359965279, -5.389897555491412, 492609224), (1359965279, 0.31535313241997104, 2107752395), '0.1958290412808642') We are sending mail with results at report@fotonower.com args[1359965262] : ((1359965262, -5.529678460617215, 492609224), (1359965262, 0.31470599384843234, 2107752395), '0.1958290412808642') We are sending mail with results at report@fotonower.com args[1359965258] : ((1359965258, -3.625401726055083, 492609224), (1359965258, -0.09902953672475415, 496442774), '0.1958290412808642') We are sending mail with results at report@fotonower.com args[1359965254] : ((1359965254, -3.1406418875714457, 492609224), (1359965254, -0.22598678552596868, 496442774), '0.1958290412808642') We are sending mail with results at report@fotonower.com refus_total : 0.1958290412808642 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23198775 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23198775_21-05-2025_09_15_02.pdf results_Auto_P23198775_21-05-2025_09_15_02.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23198775_21-05-2025_09_15_02.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23198775','results_Auto_P23198775_21-05-2025_09_15_02.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23198775_21-05-2025_09_15_02.pdf','pdf','','0.7','0.1958290412808642') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23198775

https://www.fotonower.com/image?json=false&list_photos_id=1359965362
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359965279
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359965262
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359965258
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359965254
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 19.58%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/23199392?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23199393?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23199394?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23199398?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/23199400?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23198775_21-05-2025_09_15_02.pdf.

Lien vers velours :https://www.fotonower.com/velours/23199390,23199391,23199392,23199393,23199394,23199395,23199396,23199397,23199398,23199399,23199400?tags=environnement,metal,papier,pet_clair,carton,background,autre,flou,pet_fonce,mal_croppe,pehd.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 21 May 2025 07:15:09 GMT Content-Length: 0 Connection: close X-Message-Id: Cc0-4UfgRbCL_PMMRBtbVQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359965362, 1359965279, 1359965262, 1359965258, 1359965254] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965362', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965279', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965262', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965258', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965254', None, None, None, None, None, '2919258') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.0200955867767334 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.024329423904419 time spend to save output : 0.020346879959106445 total time spend for step 9 : 7.044676303863525 step10:split_time_score Wed May 21 09:15:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('06', 5),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 21052025 23198775 Nombre de photos uploadées : 5 / 23040 (0%) 21052025 23198775 Nombre de photos taguées (types de déchets): 0 / 5 (0%) 21052025 23198775 Nombre de photos taguées (volume) : 0 / 5 (0%) elapsed_time : load_data_split_time_score 3.5762786865234375e-06 elapsed_time : order_list_meta_photo_and_scores 9.775161743164062e-06 ????? elapsed_time : fill_and_build_computed_from_old_data 0.000614166259765625 elapsed_time : insert_dashboard_record_day_entry 0.034867048263549805 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.1958290412808642 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23198775_21-05-2025_09_15_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23198775 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23198775 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'21052025': {'nb_upload': 5, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359965362, 1359965279, 1359965262, 1359965258, 1359965254] Looping around the photos to save general results len do output : 1 /23198775Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965362', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965279', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965262', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965258', None, None, None, None, None, '2919258') ('3318', None, None, None, None, None, None, None, '2919258') ('3318', '23198775', '1359965254', None, None, None, None, None, '2919258') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.015208721160888672 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3427309989929199 time spend to save output : 0.01544499397277832 total time spend for step 10 : 0.35817599296569824 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 5 set_done_treatment 108.08user 91.55system 4:44.30elapsed 70%CPU (0avgtext+0avgdata 3233212maxresident)k 1427008inputs+133496outputs (25279major+8786604minor)pagefaults 0swaps