python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3380934 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2925333'] with mtr_portfolio_ids : ['23213504'] and first list_photo_ids : [] new path : /proc/3380934/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 7 ; length of list_pids : 7 ; length of list_args : 7 time to download the photos : 1.2132112979888916 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed May 21 16:20:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 7035 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-21 16:20:33.366508: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-21 16:20:33.395312: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-21 16:20:33.397488: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff0dc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-21 16:20:33.397519: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-21 16:20:33.401581: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-21 16:20:33.656112: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3f213920 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-21 16:20:33.656175: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-21 16:20:33.657326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 16:20:33.658008: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 16:20:33.662363: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 16:20:33.665105: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 16:20:33.666500: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 16:20:33.671064: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 16:20:33.673636: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 16:20:33.682605: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 16:20:33.684705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 16:20:33.684856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 16:20:33.685866: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 16:20:33.685892: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 16:20:33.685910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 16:20:33.688230: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6470 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-21 16:20:34.130795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 16:20:34.130975: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 16:20:34.131007: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 16:20:34.131029: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 16:20:34.131058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 16:20:34.131082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 16:20:34.131103: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 16:20:34.131127: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 16:20:34.132618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 16:20:34.134326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 16:20:34.134427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 16:20:34.134448: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 16:20:34.134464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 16:20:34.134482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 16:20:34.134504: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 16:20:34.134527: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 16:20:34.134550: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 16:20:34.135918: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 16:20:34.135984: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 16:20:34.136006: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 16:20:34.136018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 16:20:34.137412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6470 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-21 16:20:46.450454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 16:20:46.657570: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 16:20:48.167558: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.168150: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.168725: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.169253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.169792: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.62G (2817927680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.170309: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.36G (2536134912 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.170845: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.12G (2282521344 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.170877: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.171601: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.171621: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.177638: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.177660: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.178169: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.178184: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.188043: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.188083: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.188628: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.188644: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.214196: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.214225: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.214734: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.214749: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.220079: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.220100: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.220606: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.220621: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-21 16:20:48.248303: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.248869: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.253963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.254485: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.290877: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.291423: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.308115: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.310408: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.340830: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.341396: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.342857: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.343393: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.348673: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.349180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.350804: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.351357: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.357433: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.357943: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.359414: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.359963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.391198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.391750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.392253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.392756: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.396172: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.396679: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.412800: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.413389: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.413895: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.414401: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.427112: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.427690: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.428235: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.428748: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.432969: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.433480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.437933: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.438443: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.450617: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.451191: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.455290: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.455838: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.456384: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.456889: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.457678: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.458189: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.469022: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.469533: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.470052: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.470556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.471067: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.471573: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.472076: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.472580: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.481834: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.482365: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.489002: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.489572: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.540122: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.540697: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.541221: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.541730: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.549654: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.550207: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.570834: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.571646: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.572420: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.573175: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.577238: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.577749: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.578254: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.578758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.579772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.579789: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-21 16:20:48.590040: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.590591: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.599517: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.600030: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.600545: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.601050: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.601562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-21 16:20:48.602066: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 7 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 9.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 31 Detection mask done ! Trying to reset tf kernel 3381736 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2379 tf kernel not reseted sub process len(results) : 7 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 7 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 7037 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.003798961639404297 nb_pixel_total : 166960 time to create 1 rle with new method : 0.008856058120727539 length of segment : 368 time for calcul the mask position with numpy : 0.0015370845794677734 nb_pixel_total : 66996 time to create 1 rle with old method : 0.07713985443115234 length of segment : 560 time for calcul the mask position with numpy : 0.002035856246948242 nb_pixel_total : 129233 time to create 1 rle with old method : 0.1442852020263672 length of segment : 459 time for calcul the mask position with numpy : 0.0013217926025390625 nb_pixel_total : 51312 time to create 1 rle with old method : 0.05737590789794922 length of segment : 472 time for calcul the mask position with numpy : 0.0010461807250976562 nb_pixel_total : 60625 time to create 1 rle with old method : 0.06959390640258789 length of segment : 267 time for calcul the mask position with numpy : 0.0003867149353027344 nb_pixel_total : 18484 time to create 1 rle with old method : 0.021932363510131836 length of segment : 163 time for calcul the mask position with numpy : 0.0003364086151123047 nb_pixel_total : 19381 time to create 1 rle with old method : 0.02242732048034668 length of segment : 248 time for calcul the mask position with numpy : 0.0004661083221435547 nb_pixel_total : 24821 time to create 1 rle with old method : 0.02841639518737793 length of segment : 240 time for calcul the mask position with numpy : 0.0006976127624511719 nb_pixel_total : 31598 time to create 1 rle with old method : 0.0360107421875 length of segment : 303 time for calcul the mask position with numpy : 0.0005724430084228516 nb_pixel_total : 32968 time to create 1 rle with old method : 0.036809682846069336 length of segment : 263 time for calcul the mask position with numpy : 0.7701132297515869 nb_pixel_total : 677367 time to create 1 rle with new method : 0.05478835105895996 length of segment : 1234 time for calcul the mask position with numpy : 0.0015568733215332031 nb_pixel_total : 98034 time to create 1 rle with old method : 0.11148357391357422 length of segment : 703 time for calcul the mask position with numpy : 0.0014307498931884766 nb_pixel_total : 46992 time to create 1 rle with old method : 0.05514335632324219 length of segment : 623 time for calcul the mask position with numpy : 0.003589153289794922 nb_pixel_total : 252991 time to create 1 rle with new method : 0.009760856628417969 length of segment : 510 time for calcul the mask position with numpy : 0.0027713775634765625 nb_pixel_total : 154770 time to create 1 rle with new method : 0.007564067840576172 length of segment : 516 time for calcul the mask position with numpy : 0.00021982192993164062 nb_pixel_total : 7114 time to create 1 rle with old method : 0.008572101593017578 length of segment : 107 time for calcul the mask position with numpy : 0.0007641315460205078 nb_pixel_total : 47561 time to create 1 rle with old method : 0.06087970733642578 length of segment : 267 time for calcul the mask position with numpy : 0.00133514404296875 nb_pixel_total : 82905 time to create 1 rle with old method : 0.09698820114135742 length of segment : 351 time for calcul the mask position with numpy : 0.006017923355102539 nb_pixel_total : 263373 time to create 1 rle with new method : 0.018448829650878906 length of segment : 598 time for calcul the mask position with numpy : 0.0027544498443603516 nb_pixel_total : 117418 time to create 1 rle with old method : 0.2112276554107666 length of segment : 439 time for calcul the mask position with numpy : 0.023193359375 nb_pixel_total : 915132 time to create 1 rle with new method : 0.4928276538848877 length of segment : 1479 time for calcul the mask position with numpy : 0.0003693103790283203 nb_pixel_total : 17683 time to create 1 rle with old method : 0.020700454711914062 length of segment : 224 time for calcul the mask position with numpy : 0.0026280879974365234 nb_pixel_total : 169849 time to create 1 rle with new method : 0.007831335067749023 length of segment : 544 time for calcul the mask position with numpy : 0.0008933544158935547 nb_pixel_total : 41365 time to create 1 rle with old method : 0.049818992614746094 length of segment : 268 time for calcul the mask position with numpy : 0.00036025047302246094 nb_pixel_total : 18532 time to create 1 rle with old method : 0.02229142189025879 length of segment : 191 time for calcul the mask position with numpy : 0.0006721019744873047 nb_pixel_total : 38360 time to create 1 rle with old method : 0.0462336540222168 length of segment : 283 time for calcul the mask position with numpy : 0.0036873817443847656 nb_pixel_total : 244368 time to create 1 rle with new method : 0.009533405303955078 length of segment : 612 time for calcul the mask position with numpy : 0.0018472671508789062 nb_pixel_total : 52123 time to create 1 rle with old method : 0.06270360946655273 length of segment : 698 time for calcul the mask position with numpy : 0.0008106231689453125 nb_pixel_total : 29987 time to create 1 rle with old method : 0.035265445709228516 length of segment : 408 time for calcul the mask position with numpy : 0.002965688705444336 nb_pixel_total : 84738 time to create 1 rle with old method : 0.1006155014038086 length of segment : 491 time for calcul the mask position with numpy : 0.00018715858459472656 nb_pixel_total : 7736 time to create 1 rle with old method : 0.009190082550048828 length of segment : 114 time for calcul the mask position with numpy : 0.002042531967163086 nb_pixel_total : 145554 time to create 1 rle with old method : 0.17465877532958984 length of segment : 318 time for calcul the mask position with numpy : 0.00238037109375 nb_pixel_total : 149694 time to create 1 rle with old method : 0.17679524421691895 length of segment : 333 time for calcul the mask position with numpy : 0.0032956600189208984 nb_pixel_total : 246939 time to create 1 rle with new method : 0.0077877044677734375 length of segment : 620 time for calcul the mask position with numpy : 0.0013561248779296875 nb_pixel_total : 39817 time to create 1 rle with old method : 0.06606388092041016 length of segment : 217 time for calcul the mask position with numpy : 0.0007603168487548828 nb_pixel_total : 16858 time to create 1 rle with old method : 0.02062082290649414 length of segment : 208 time for calcul the mask position with numpy : 0.0023152828216552734 nb_pixel_total : 92841 time to create 1 rle with old method : 0.1085212230682373 length of segment : 349 time for calcul the mask position with numpy : 0.0028760433197021484 nb_pixel_total : 71358 time to create 1 rle with old method : 0.08300495147705078 length of segment : 1104 time for calcul the mask position with numpy : 0.0008549690246582031 nb_pixel_total : 24435 time to create 1 rle with old method : 0.028959274291992188 length of segment : 166 time for calcul the mask position with numpy : 0.004896402359008789 nb_pixel_total : 203003 time to create 1 rle with new method : 0.0070667266845703125 length of segment : 567 time for calcul the mask position with numpy : 0.002092123031616211 nb_pixel_total : 50064 time to create 1 rle with old method : 0.06251382827758789 length of segment : 244 time for calcul the mask position with numpy : 0.0054776668548583984 nb_pixel_total : 187412 time to create 1 rle with new method : 0.0075397491455078125 length of segment : 521 time for calcul the mask position with numpy : 0.002132892608642578 nb_pixel_total : 106340 time to create 1 rle with old method : 0.12327861785888672 length of segment : 439 time for calcul the mask position with numpy : 0.002081632614135742 nb_pixel_total : 50738 time to create 1 rle with old method : 0.05843067169189453 length of segment : 444 time for calcul the mask position with numpy : 0.0008571147918701172 nb_pixel_total : 34672 time to create 1 rle with old method : 0.03937888145446777 length of segment : 313 time for calcul the mask position with numpy : 0.006270647048950195 nb_pixel_total : 267761 time to create 1 rle with new method : 0.009438276290893555 length of segment : 689 time for calcul the mask position with numpy : 0.005399465560913086 nb_pixel_total : 204015 time to create 1 rle with new method : 0.008941650390625 length of segment : 645 time for calcul the mask position with numpy : 0.0012106895446777344 nb_pixel_total : 50401 time to create 1 rle with old method : 0.059366464614868164 length of segment : 310 time for calcul the mask position with numpy : 0.001325368881225586 nb_pixel_total : 58201 time to create 1 rle with old method : 0.06702542304992676 length of segment : 298 time for calcul the mask position with numpy : 0.0032155513763427734 nb_pixel_total : 145017 time to create 1 rle with old method : 0.1880953311920166 length of segment : 469 time for calcul the mask position with numpy : 0.002336740493774414 nb_pixel_total : 116703 time to create 1 rle with old method : 0.13474726676940918 length of segment : 400 time for calcul the mask position with numpy : 0.0004930496215820312 nb_pixel_total : 13506 time to create 1 rle with old method : 0.016178369522094727 length of segment : 131 time for calcul the mask position with numpy : 0.003121614456176758 nb_pixel_total : 74590 time to create 1 rle with old method : 0.08731746673583984 length of segment : 709 time for calcul the mask position with numpy : 0.00043845176696777344 nb_pixel_total : 29448 time to create 1 rle with old method : 0.034438133239746094 length of segment : 250 time for calcul the mask position with numpy : 0.0023603439331054688 nb_pixel_total : 89963 time to create 1 rle with old method : 0.10382938385009766 length of segment : 479 time for calcul the mask position with numpy : 0.0008807182312011719 nb_pixel_total : 30259 time to create 1 rle with old method : 0.03647565841674805 length of segment : 157 time for calcul the mask position with numpy : 0.0009467601776123047 nb_pixel_total : 34290 time to create 1 rle with old method : 0.04076838493347168 length of segment : 243 time for calcul the mask position with numpy : 0.005468606948852539 nb_pixel_total : 268137 time to create 1 rle with new method : 0.00921630859375 length of segment : 646 time for calcul the mask position with numpy : 0.0016672611236572266 nb_pixel_total : 83483 time to create 1 rle with old method : 0.09359908103942871 length of segment : 484 time for calcul the mask position with numpy : 0.005620002746582031 nb_pixel_total : 162376 time to create 1 rle with new method : 0.006861209869384766 length of segment : 504 time for calcul the mask position with numpy : 0.002824544906616211 nb_pixel_total : 68476 time to create 1 rle with old method : 0.08277320861816406 length of segment : 629 time for calcul the mask position with numpy : 0.0017943382263183594 nb_pixel_total : 65476 time to create 1 rle with old method : 0.08004283905029297 length of segment : 336 time for calcul the mask position with numpy : 0.0010418891906738281 nb_pixel_total : 36208 time to create 1 rle with old method : 0.042606353759765625 length of segment : 183 time for calcul the mask position with numpy : 0.0008006095886230469 nb_pixel_total : 13800 time to create 1 rle with old method : 0.016106128692626953 length of segment : 136 time for calcul the mask position with numpy : 0.0008873939514160156 nb_pixel_total : 38769 time to create 1 rle with old method : 0.04719424247741699 length of segment : 192 time for calcul the mask position with numpy : 0.0005741119384765625 nb_pixel_total : 23344 time to create 1 rle with old method : 0.029618024826049805 length of segment : 131 time for calcul the mask position with numpy : 0.0005254745483398438 nb_pixel_total : 26094 time to create 1 rle with old method : 0.031206607818603516 length of segment : 132 time for calcul the mask position with numpy : 0.003151416778564453 nb_pixel_total : 161314 time to create 1 rle with new method : 0.0056188106536865234 length of segment : 499 time for calcul the mask position with numpy : 0.004555702209472656 nb_pixel_total : 213398 time to create 1 rle with new method : 0.009649991989135742 length of segment : 601 time for calcul the mask position with numpy : 0.0045604705810546875 nb_pixel_total : 196280 time to create 1 rle with new method : 0.008331775665283203 length of segment : 421 time for calcul the mask position with numpy : 0.0018892288208007812 nb_pixel_total : 52280 time to create 1 rle with old method : 0.06123185157775879 length of segment : 380 time for calcul the mask position with numpy : 0.004225730895996094 nb_pixel_total : 139847 time to create 1 rle with old method : 0.15952825546264648 length of segment : 560 time for calcul the mask position with numpy : 0.0037729740142822266 nb_pixel_total : 179924 time to create 1 rle with new method : 0.009555339813232422 length of segment : 445 time for calcul the mask position with numpy : 0.0003616809844970703 nb_pixel_total : 18145 time to create 1 rle with old method : 0.021671533584594727 length of segment : 217 time for calcul the mask position with numpy : 0.0011553764343261719 nb_pixel_total : 42743 time to create 1 rle with old method : 0.05097317695617676 length of segment : 385 time for calcul the mask position with numpy : 0.0004241466522216797 nb_pixel_total : 18887 time to create 1 rle with old method : 0.022949934005737305 length of segment : 227 time for calcul the mask position with numpy : 0.0020978450775146484 nb_pixel_total : 103766 time to create 1 rle with old method : 0.12064051628112793 length of segment : 331 time for calcul the mask position with numpy : 0.002473115921020508 nb_pixel_total : 183035 time to create 1 rle with new method : 0.00636601448059082 length of segment : 426 time for calcul the mask position with numpy : 0.0016443729400634766 nb_pixel_total : 73329 time to create 1 rle with old method : 0.09311294555664062 length of segment : 272 time for calcul the mask position with numpy : 0.004397392272949219 nb_pixel_total : 150289 time to create 1 rle with new method : 0.01240086555480957 length of segment : 1126 time for calcul the mask position with numpy : 0.00037026405334472656 nb_pixel_total : 14693 time to create 1 rle with old method : 0.01737213134765625 length of segment : 166 time for calcul the mask position with numpy : 0.0047893524169921875 nb_pixel_total : 65179 time to create 1 rle with old method : 0.08081364631652832 length of segment : 612 time for calcul the mask position with numpy : 0.0046770572662353516 nb_pixel_total : 152010 time to create 1 rle with new method : 0.006727933883666992 length of segment : 509 time for calcul the mask position with numpy : 0.007232189178466797 nb_pixel_total : 146539 time to create 1 rle with old method : 0.1800074577331543 length of segment : 721 time for calcul the mask position with numpy : 0.004177093505859375 nb_pixel_total : 129031 time to create 1 rle with old method : 0.1494297981262207 length of segment : 643 time for calcul the mask position with numpy : 0.0027000904083251953 nb_pixel_total : 85699 time to create 1 rle with old method : 0.09763503074645996 length of segment : 461 time for calcul the mask position with numpy : 0.0010933876037597656 nb_pixel_total : 24389 time to create 1 rle with old method : 0.028670787811279297 length of segment : 380 time for calcul the mask position with numpy : 0.004037380218505859 nb_pixel_total : 138897 time to create 1 rle with old method : 0.18712902069091797 length of segment : 503 time for calcul the mask position with numpy : 0.0041844844818115234 nb_pixel_total : 101887 time to create 1 rle with old method : 0.11720132827758789 length of segment : 670 time for calcul the mask position with numpy : 0.0019290447235107422 nb_pixel_total : 50982 time to create 1 rle with old method : 0.059999942779541016 length of segment : 453 time for calcul the mask position with numpy : 0.005093097686767578 nb_pixel_total : 84406 time to create 1 rle with old method : 0.09935617446899414 length of segment : 840 time for calcul the mask position with numpy : 0.017104387283325195 nb_pixel_total : 223598 time to create 1 rle with new method : 0.10556483268737793 length of segment : 1212 time for calcul the mask position with numpy : 0.0009641647338867188 nb_pixel_total : 33173 time to create 1 rle with old method : 0.037921905517578125 length of segment : 190 time for calcul the mask position with numpy : 0.0012526512145996094 nb_pixel_total : 45157 time to create 1 rle with old method : 0.05388212203979492 length of segment : 375 time for calcul the mask position with numpy : 0.0010447502136230469 nb_pixel_total : 39627 time to create 1 rle with old method : 0.04807400703430176 length of segment : 274 time for calcul the mask position with numpy : 0.006279945373535156 nb_pixel_total : 166465 time to create 1 rle with new method : 0.01262211799621582 length of segment : 964 time for calcul the mask position with numpy : 0.0005624294281005859 nb_pixel_total : 14818 time to create 1 rle with old method : 0.017867565155029297 length of segment : 154 time spent for convertir_results : 13.650266647338867 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 97 chid ids of type : 3594 Number RLEs to save : 43016 save missing photos in datou_result : time spend for datou_step_exec : 64.87519979476929 time spend to save output : 2.674255847930908 total time spend for step 1 : 67.5494556427002 step2:crop_condition Wed May 21 16:21:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 7 ! batch 1 Loaded 97 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 82 About to insert : list_path_to_insert length 82 new photo from crops ! About to upload 82 photos upload in portfolio : 3736932 init cache_photo without model_param we have 82 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747837339_3380934 we have uploaded 82 photos in the portfolio 3736932 time of upload the photos Elapsed time : 22.70668888092041 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747837365_3380934 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.7952253818511963 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747837367_3380934 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5713200569152832 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747837372_3380934 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.8361423015594482 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747837376_3380934 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8950886726379395 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1360043487, 1360043323, 1360043304, 1360043281, 1360043256, 1360043244, 1360043131] Looping around the photos to save general results len do output : 97 /1360054939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360054999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360055074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043487', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043323', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043304', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043281', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043256', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043244', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043131', None, None, None, None, None, '2925333') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 298 time used for this insertion : 0.039163827896118164 save_final save missing photos in datou_result : time spend for datou_step_exec : 79.16117238998413 time spend to save output : 0.04216933250427246 total time spend for step 2 : 79.2033417224884 step3:rle_unique_nms_with_priority Wed May 21 16:22:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 97 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 6.776727914810181 time for calcul the mask position with numpy : 0.3797318935394287 nb_pixel_total : 6410689 time to create 1 rle with new method : 0.9582529067993164 time for calcul the mask position with numpy : 0.026413679122924805 nb_pixel_total : 47561 time to create 1 rle with old method : 0.054639577865600586 time for calcul the mask position with numpy : 0.027692794799804688 nb_pixel_total : 7114 time to create 1 rle with old method : 0.008521556854248047 time for calcul the mask position with numpy : 0.028290271759033203 nb_pixel_total : 154770 time to create 1 rle with new method : 0.6263260841369629 time for calcul the mask position with numpy : 0.03385281562805176 nb_pixel_total : 252991 time to create 1 rle with new method : 0.7228245735168457 time for calcul the mask position with numpy : 0.03636527061462402 nb_pixel_total : 46992 time to create 1 rle with old method : 0.06034278869628906 time for calcul the mask position with numpy : 0.04450392723083496 nb_pixel_total : 95413 time to create 1 rle with old method : 0.15986156463623047 time for calcul the mask position with numpy : 0.045952796936035156 nb_pixel_total : 677367 time to create 1 rle with new method : 0.8859834671020508 time for calcul the mask position with numpy : 0.03446054458618164 nb_pixel_total : 32968 time to create 1 rle with old method : 0.03818678855895996 time for calcul the mask position with numpy : 0.03550076484680176 nb_pixel_total : 31598 time to create 1 rle with old method : 0.035900115966796875 time for calcul the mask position with numpy : 0.03445792198181152 nb_pixel_total : 24821 time to create 1 rle with old method : 0.028491735458374023 time for calcul the mask position with numpy : 0.035009145736694336 nb_pixel_total : 19381 time to create 1 rle with old method : 0.022208690643310547 time for calcul the mask position with numpy : 0.03968930244445801 nb_pixel_total : 17609 time to create 1 rle with old method : 0.020078420639038086 time for calcul the mask position with numpy : 0.03982138633728027 nb_pixel_total : 60625 time to create 1 rle with old method : 0.06968379020690918 time for calcul the mask position with numpy : 0.030869483947753906 nb_pixel_total : 51312 time to create 1 rle with old method : 0.05710124969482422 time for calcul the mask position with numpy : 0.0304872989654541 nb_pixel_total : 129233 time to create 1 rle with old method : 0.14317631721496582 time for calcul the mask position with numpy : 0.02652454376220703 nb_pixel_total : 66996 time to create 1 rle with old method : 0.07718992233276367 time for calcul the mask position with numpy : 0.027881860733032227 nb_pixel_total : 166960 time to create 1 rle with new method : 0.6501507759094238 create new chi : 5.743056058883667 time to delete rle : 0.024661779403686523 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 16692 TO DO : save crop sub photo not yet done ! save time : 1.0351893901824951 nb_obj : 17 nb_hashtags : 2 time to prepare the origin masks : 7.274499416351318 time for calcul the mask position with numpy : 0.44765400886535645 nb_pixel_total : 6085059 time to create 1 rle with new method : 0.5755658149719238 time for calcul the mask position with numpy : 0.03180670738220215 nb_pixel_total : 246939 time to create 1 rle with new method : 0.7244019508361816 time for calcul the mask position with numpy : 0.03545546531677246 nb_pixel_total : 4361 time to create 1 rle with old method : 0.005918979644775391 time for calcul the mask position with numpy : 0.0324556827545166 nb_pixel_total : 145554 time to create 1 rle with old method : 0.16768693923950195 time for calcul the mask position with numpy : 0.03142857551574707 nb_pixel_total : 7736 time to create 1 rle with old method : 0.008728742599487305 time for calcul the mask position with numpy : 0.033368587493896484 nb_pixel_total : 84738 time to create 1 rle with old method : 0.0982813835144043 time for calcul the mask position with numpy : 0.03330349922180176 nb_pixel_total : 29987 time to create 1 rle with old method : 0.036412715911865234 time for calcul the mask position with numpy : 0.026426315307617188 nb_pixel_total : 52123 time to create 1 rle with old method : 0.05933880805969238 time for calcul the mask position with numpy : 0.026561260223388672 nb_pixel_total : 142785 time to create 1 rle with old method : 0.1585681438446045 time for calcul the mask position with numpy : 0.02384495735168457 nb_pixel_total : 38360 time to create 1 rle with old method : 0.043118953704833984 time for calcul the mask position with numpy : 0.024515151977539062 nb_pixel_total : 18532 time to create 1 rle with old method : 0.020332813262939453 time for calcul the mask position with numpy : 0.023815155029296875 nb_pixel_total : 41365 time to create 1 rle with old method : 0.04560732841491699 time for calcul the mask position with numpy : 0.02412581443786621 nb_pixel_total : 58149 time to create 1 rle with old method : 0.08887338638305664 time for calcul the mask position with numpy : 0.027087688446044922 nb_pixel_total : 17683 time to create 1 rle with old method : 0.02003002166748047 time for calcul the mask position with numpy : 0.030229568481445312 nb_pixel_total : 857333 time to create 1 rle with new method : 0.4958195686340332 time for calcul the mask position with numpy : 0.02378678321838379 nb_pixel_total : 117418 time to create 1 rle with old method : 0.13042879104614258 time for calcul the mask position with numpy : 0.0265347957611084 nb_pixel_total : 263373 time to create 1 rle with new method : 0.4219508171081543 time for calcul the mask position with numpy : 0.026507139205932617 nb_pixel_total : 82905 time to create 1 rle with old method : 0.09409308433532715 create new chi : 4.259527683258057 time to delete rle : 0.002233743667602539 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++++++++++Number RLEs to save : 16913 TO DO : save crop sub photo not yet done ! save time : 1.064011812210083 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 4.727328538894653 time for calcul the mask position with numpy : 0.38794946670532227 nb_pixel_total : 7452050 time to create 1 rle with new method : 0.5221080780029297 time for calcul the mask position with numpy : 0.025665283203125 nb_pixel_total : 34672 time to create 1 rle with old method : 0.04037046432495117 time for calcul the mask position with numpy : 0.026744365692138672 nb_pixel_total : 50738 time to create 1 rle with old method : 0.05936551094055176 time for calcul the mask position with numpy : 0.026934146881103516 nb_pixel_total : 71152 time to create 1 rle with old method : 0.08227252960205078 time for calcul the mask position with numpy : 0.027350425720214844 nb_pixel_total : 187412 time to create 1 rle with new method : 0.5020177364349365 time for calcul the mask position with numpy : 0.024254322052001953 nb_pixel_total : 50064 time to create 1 rle with old method : 0.05524611473083496 time for calcul the mask position with numpy : 0.02489161491394043 nb_pixel_total : 203003 time to create 1 rle with new method : 1.251840591430664 time for calcul the mask position with numpy : 0.025786876678466797 nb_pixel_total : 24435 time to create 1 rle with old method : 0.030154705047607422 time for calcul the mask position with numpy : 0.026810884475708008 nb_pixel_total : 71358 time to create 1 rle with old method : 0.08264780044555664 time for calcul the mask position with numpy : 0.026831865310668945 nb_pixel_total : 92841 time to create 1 rle with old method : 0.10776281356811523 time for calcul the mask position with numpy : 0.025243043899536133 nb_pixel_total : 16858 time to create 1 rle with old method : 0.01969313621520996 time for calcul the mask position with numpy : 0.026348352432250977 nb_pixel_total : 39817 time to create 1 rle with old method : 0.04618692398071289 create new chi : 3.574467182159424 time to delete rle : 0.0017344951629638672 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 10962 TO DO : save crop sub photo not yet done ! save time : 0.7024853229522705 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 8.257309198379517 time for calcul the mask position with numpy : 0.7452590465545654 nb_pixel_total : 7215938 time to create 1 rle with new method : 0.889885663986206 time for calcul the mask position with numpy : 0.025139570236206055 nb_pixel_total : 30259 time to create 1 rle with old method : 0.03461861610412598 time for calcul the mask position with numpy : 0.02391362190246582 nb_pixel_total : 88561 time to create 1 rle with old method : 0.0982503890991211 time for calcul the mask position with numpy : 0.024661779403686523 nb_pixel_total : 29448 time to create 1 rle with old method : 0.034044504165649414 time for calcul the mask position with numpy : 0.02499842643737793 nb_pixel_total : 74590 time to create 1 rle with old method : 0.08417415618896484 time for calcul the mask position with numpy : 0.024147510528564453 nb_pixel_total : 13506 time to create 1 rle with old method : 0.01575303077697754 time for calcul the mask position with numpy : 0.0278017520904541 nb_pixel_total : 116703 time to create 1 rle with old method : 0.13440394401550293 time for calcul the mask position with numpy : 0.02609705924987793 nb_pixel_total : 145017 time to create 1 rle with old method : 0.16669750213623047 time for calcul the mask position with numpy : 0.02632880210876465 nb_pixel_total : 58201 time to create 1 rle with old method : 0.06705570220947266 time for calcul the mask position with numpy : 0.025948762893676758 nb_pixel_total : 50401 time to create 1 rle with old method : 0.05861687660217285 time for calcul the mask position with numpy : 0.027781963348388672 nb_pixel_total : 204015 time to create 1 rle with new method : 0.670499324798584 time for calcul the mask position with numpy : 0.027039051055908203 nb_pixel_total : 267761 time to create 1 rle with new method : 0.761152982711792 create new chi : 4.139363527297974 time to delete rle : 0.0018527507781982422 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 11116 TO DO : save crop sub photo not yet done ! save time : 0.7363121509552002 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 5.96433162689209 time for calcul the mask position with numpy : 0.6803250312805176 nb_pixel_total : 7312633 time to create 1 rle with new method : 0.7964704036712646 time for calcul the mask position with numpy : 0.025049209594726562 nb_pixel_total : 161314 time to create 1 rle with new method : 0.780214786529541 time for calcul the mask position with numpy : 0.023929357528686523 nb_pixel_total : 26094 time to create 1 rle with old method : 0.028420448303222656 time for calcul the mask position with numpy : 0.023495197296142578 nb_pixel_total : 23344 time to create 1 rle with old method : 0.02570629119873047 time for calcul the mask position with numpy : 0.023869991302490234 nb_pixel_total : 38769 time to create 1 rle with old method : 0.04299426078796387 time for calcul the mask position with numpy : 0.023999929428100586 nb_pixel_total : 13800 time to create 1 rle with old method : 0.015511512756347656 time for calcul the mask position with numpy : 0.024358272552490234 nb_pixel_total : 36208 time to create 1 rle with old method : 0.040906667709350586 time for calcul the mask position with numpy : 0.02427053451538086 nb_pixel_total : 65476 time to create 1 rle with old method : 0.07163071632385254 time for calcul the mask position with numpy : 0.024738788604736328 nb_pixel_total : 68476 time to create 1 rle with old method : 0.07810568809509277 time for calcul the mask position with numpy : 0.02529001235961914 nb_pixel_total : 162376 time to create 1 rle with new method : 0.6549780368804932 time for calcul the mask position with numpy : 0.029239654541015625 nb_pixel_total : 83483 time to create 1 rle with old method : 0.09460020065307617 time for calcul the mask position with numpy : 0.026829957962036133 nb_pixel_total : 268137 time to create 1 rle with new method : 0.8107306957244873 time for calcul the mask position with numpy : 0.03186488151550293 nb_pixel_total : 34290 time to create 1 rle with old method : 0.039580345153808594 create new chi : 4.589996814727783 time to delete rle : 0.0016160011291503906 batch 1 Loaded 25 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 10390 TO DO : save crop sub photo not yet done ! save time : 0.6884384155273438 nb_obj : 13 nb_hashtags : 3 time to prepare the origin masks : 5.167744398117065 time for calcul the mask position with numpy : 0.6947293281555176 nb_pixel_total : 7101870 time to create 1 rle with new method : 1.0297977924346924 time for calcul the mask position with numpy : 0.02579975128173828 nb_pixel_total : 14693 time to create 1 rle with old method : 0.01694035530090332 time for calcul the mask position with numpy : 0.02520465850830078 nb_pixel_total : 133889 time to create 1 rle with old method : 0.17440366744995117 time for calcul the mask position with numpy : 0.026267290115356445 nb_pixel_total : 73329 time to create 1 rle with old method : 0.08535933494567871 time for calcul the mask position with numpy : 0.03025364875793457 nb_pixel_total : 5513 time to create 1 rle with old method : 0.007294416427612305 time for calcul the mask position with numpy : 0.02857208251953125 nb_pixel_total : 103766 time to create 1 rle with old method : 0.12427043914794922 time for calcul the mask position with numpy : 0.02594280242919922 nb_pixel_total : 18887 time to create 1 rle with old method : 0.022037982940673828 time for calcul the mask position with numpy : 0.026798486709594727 nb_pixel_total : 42743 time to create 1 rle with old method : 0.05049419403076172 time for calcul the mask position with numpy : 0.029278039932250977 nb_pixel_total : 17981 time to create 1 rle with old method : 0.023407459259033203 time for calcul the mask position with numpy : 0.026159048080444336 nb_pixel_total : 179924 time to create 1 rle with new method : 0.9502105712890625 time for calcul the mask position with numpy : 0.029227018356323242 nb_pixel_total : 139847 time to create 1 rle with old method : 0.15784931182861328 time for calcul the mask position with numpy : 0.025351285934448242 nb_pixel_total : 52280 time to create 1 rle with old method : 0.06325268745422363 time for calcul the mask position with numpy : 0.028784513473510742 nb_pixel_total : 196280 time to create 1 rle with new method : 0.8727207183837891 time for calcul the mask position with numpy : 0.028087854385375977 nb_pixel_total : 213398 time to create 1 rle with new method : 0.8316755294799805 create new chi : 5.592413663864136 time to delete rle : 0.0018694400787353516 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 12560 TO DO : save crop sub photo not yet done ! save time : 0.8471360206604004 nb_obj : 16 nb_hashtags : 3 time to prepare the origin masks : 7.363649606704712 time for calcul the mask position with numpy : 0.4961435794830322 nb_pixel_total : 6923924 time to create 1 rle with new method : 2.4770498275756836 time for calcul the mask position with numpy : 0.024553537368774414 nb_pixel_total : 14818 time to create 1 rle with old method : 0.01672840118408203 time for calcul the mask position with numpy : 0.026727914810180664 nb_pixel_total : 166465 time to create 1 rle with new method : 0.8421950340270996 time for calcul the mask position with numpy : 0.030319690704345703 nb_pixel_total : 39627 time to create 1 rle with old method : 0.04998517036437988 time for calcul the mask position with numpy : 0.028101682662963867 nb_pixel_total : 6474 time to create 1 rle with old method : 0.007877826690673828 time for calcul the mask position with numpy : 0.03202319145202637 nb_pixel_total : 33173 time to create 1 rle with old method : 0.043436527252197266 time for calcul the mask position with numpy : 0.02902841567993164 nb_pixel_total : 130900 time to create 1 rle with old method : 0.1540818214416504 time for calcul the mask position with numpy : 0.028316497802734375 nb_pixel_total : 84406 time to create 1 rle with old method : 0.09790778160095215 time for calcul the mask position with numpy : 0.028012514114379883 nb_pixel_total : 50982 time to create 1 rle with old method : 0.07529449462890625 time for calcul the mask position with numpy : 0.03308439254760742 nb_pixel_total : 101887 time to create 1 rle with old method : 0.11842656135559082 time for calcul the mask position with numpy : 0.028182268142700195 nb_pixel_total : 138897 time to create 1 rle with old method : 0.1622636318206787 time for calcul the mask position with numpy : 0.028310775756835938 nb_pixel_total : 24389 time to create 1 rle with old method : 0.02832198143005371 time for calcul the mask position with numpy : 0.026893138885498047 nb_pixel_total : 85699 time to create 1 rle with old method : 0.09800982475280762 time for calcul the mask position with numpy : 0.030355453491210938 nb_pixel_total : 129031 time to create 1 rle with old method : 0.14684128761291504 time for calcul the mask position with numpy : 0.02591991424560547 nb_pixel_total : 146539 time to create 1 rle with old method : 0.16457700729370117 time for calcul the mask position with numpy : 0.026676177978515625 nb_pixel_total : 152010 time to create 1 rle with new method : 0.7458069324493408 time for calcul the mask position with numpy : 0.02707505226135254 nb_pixel_total : 65179 time to create 1 rle with old method : 0.07282900810241699 create new chi : 6.3591718673706055 time to delete rle : 0.002458333969116211 batch 1 Loaded 33 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++Number RLEs to save : 19116 TO DO : save crop sub photo not yet done ! save time : 1.2350637912750244 map_output_result : {1360043487: (0.0, 'Should be the crop_list due to order', 0), 1360043323: (0.0, 'Should be the crop_list due to order', 0), 1360043304: (0.0, 'Should be the crop_list due to order', 0), 1360043281: (0.0, 'Should be the crop_list due to order', 0), 1360043256: (0.0, 'Should be the crop_list due to order', 0), 1360043244: (0.0, 'Should be the crop_list due to order', 0), 1360043131: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1360043487, 1360043323, 1360043304, 1360043281, 1360043256, 1360043244, 1360043131] Looping around the photos to save general results len do output : 7 /1360043487.Didn't retrieve data . /1360043323.Didn't retrieve data . /1360043304.Didn't retrieve data . /1360043281.Didn't retrieve data . /1360043256.Didn't retrieve data . /1360043244.Didn't retrieve data . /1360043131.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043487', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043323', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043304', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043281', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043256', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043244', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043131', None, None, None, None, None, '2925333') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.015924453735351562 save_final save missing photos in datou_result : time spend for datou_step_exec : 87.19389271736145 time spend to save output : 0.02006697654724121 total time spend for step 3 : 87.21395969390869 step4:ventilate_hashtags_in_portfolio Wed May 21 16:24:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23213504 get user id for portfolio 23213504 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23213504 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_clair','autre','carton','background','environnement','metal','pehd','mal_croppe','pet_fonce','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23213504 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_clair','autre','carton','background','environnement','metal','pehd','mal_croppe','pet_fonce','flou')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23213504 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_clair','autre','carton','background','environnement','metal','pehd','mal_croppe','pet_fonce','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23214508,23214509,23214510,23214511,23214512,23214513,23214514,23214515,23214516,23214517,23214518?tags=papier,pet_clair,autre,carton,background,environnement,metal,pehd,mal_croppe,pet_fonce,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1360043487, 1360043323, 1360043304, 1360043281, 1360043256, 1360043244, 1360043131] Looping around the photos to save general results len do output : 1 /23213504. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043487', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043323', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043304', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043281', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043256', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043244', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043131', None, None, None, None, None, '2925333') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.016399621963500977 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6938316822052002 time spend to save output : 0.016672134399414062 total time spend for step 4 : 1.7105038166046143 step5:final Wed May 21 16:24:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1360043487: ('0.16463150697200177',), 1360043323: ('0.16463150697200177',), 1360043304: ('0.16463150697200177',), 1360043281: ('0.16463150697200177',), 1360043256: ('0.16463150697200177',), 1360043244: ('0.16463150697200177',), 1360043131: ('0.16463150697200177',)} new output for save of step final : {1360043487: ('0.16463150697200177',), 1360043323: ('0.16463150697200177',), 1360043304: ('0.16463150697200177',), 1360043281: ('0.16463150697200177',), 1360043256: ('0.16463150697200177',), 1360043244: ('0.16463150697200177',), 1360043131: ('0.16463150697200177',)} [1360043487, 1360043323, 1360043304, 1360043281, 1360043256, 1360043244, 1360043131] Looping around the photos to save general results len do output : 7 /1360043487.Didn't retrieve data . /1360043323.Didn't retrieve data . /1360043304.Didn't retrieve data . /1360043281.Didn't retrieve data . /1360043256.Didn't retrieve data . /1360043244.Didn't retrieve data . /1360043131.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043487', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043323', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043304', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043281', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043256', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043244', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043131', None, None, None, None, None, '2925333') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.013742923736572266 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12936687469482422 time spend to save output : 0.014294147491455078 total time spend for step 5 : 0.1436610221862793 step6:blur_detection Wed May 21 16:24:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e.jpg resize: (2160, 3840) 1360043487 -5.850898593404707 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66.jpg resize: (2160, 3840) 1360043323 -5.574316014100544 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2.jpg resize: (2160, 3840) 1360043304 -6.604102638031094 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f.jpg resize: (2160, 3840) 1360043281 -6.41228851466266 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5.jpg resize: (2160, 3840) 1360043256 -6.289160162103518 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c.jpg resize: (2160, 3840) 1360043244 -5.600830161529649 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741.jpg resize: (2160, 3840) 1360043131 -5.971320772860409 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696738_0.png resize: (248, 96) 1360054939 -1.2886691748764902 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696734_0.png resize: (448, 411) 1360054940 -3.02630454724849 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696739_0.png resize: (240, 140) 1360054941 -2.7411048843472257 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696748_0.png resize: (260, 272) 1360054942 -2.9962827186172953 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696742_0.png resize: (1234, 837) 1360054943 -3.86795346249685 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696744_0.png resize: (382, 334) 1360054944 -3.0569373250337084 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696733_0.png resize: (471, 309) 1360054945 -3.3427497210217374 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696732_0.png resize: (362, 591) 1360054946 -1.2219562278362648 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696743_0.png resize: (459, 423) 1360054947 -2.820192841510818 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696740_0.png resize: (299, 258) 1360054949 -1.3167369059584906 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696736_0.png resize: (221, 477) 1360054950 -2.879756605371823 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696746_0.png resize: (516, 485) 1360054951 -3.7716392163600205 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696741_0.png resize: (255, 172) 1360054952 -3.72126962580709 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696737_0.png resize: (162, 201) 1360054953 -3.8890599773322783 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696735_0.png resize: (468, 202) 1360054954 -4.0837335538896635 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696765_0.png resize: (608, 468) 1360054955 -2.526704489868802 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696764_0.png resize: (323, 694) 1360054956 -3.5533844565421746 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696760_0.png resize: (342, 216) 1360054957 -3.77000227975191 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696754_0.png resize: (492, 548) 1360054958 -3.6530094671084226 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696752_0.png resize: (1148, 1391) 1360054959 -4.513515384485881 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696756_0.png resize: (191, 157) 1360054960 -2.229622621108607 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696753_0.png resize: (224, 113) 1360054961 -2.2962868739948825 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696750_0.png resize: (598, 716) 1360054962 -3.8714199959813813 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696749_0.png resize: (329, 362) 1360054963 -1.9880528327019347 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696755_0.png resize: (257, 214) 1360054964 -2.8664989728495627 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696758_0.png resize: (612, 593) 1360054965 -3.3620675241747664 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696762_0.png resize: (114, 107) 1360054966 -3.58456947518563 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696761_0.png resize: (377, 442) 1360054967 -3.930689776255976 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696759_0.png resize: (558, 270) 1360054968 -3.438650806599687 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696757_0.png resize: (192, 308) 1360054969 -4.077845005362334 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696751_0.png resize: (433, 449) 1360054970 -3.7845467437320246 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696769_0.png resize: (468, 360) 1360054971 -3.460709909746157 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696771_0.png resize: (380, 768) 1360054972 -3.629396629553486 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696770_0.png resize: (165, 247) 1360054973 -2.1101880454047848 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696772_0.png resize: (242, 395) 1360054974 -3.7146148769967104 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696768_0.png resize: (292, 424) 1360054975 -4.072432267257133 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696775_0.png resize: (365, 355) 1360054976 -3.3347327115933667 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696773_0.png resize: (499, 617) 1360054977 -4.9504943200522 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696776_0.png resize: (310, 169) 1360054978 -4.718838476747 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696766_0.png resize: (212, 236) 1360054979 -0.8183333113627872 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696784_0.png resize: (535, 347) 1360054980 -1.8812141314817084 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696783_0.png resize: (131, 142) 1360054981 -3.790989500520795 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696779_0.png resize: (310, 234) 1360054982 -1.7676681813412118 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696785_0.png resize: (249, 149) 1360054983 -2.8880571380006477 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696777_0.png resize: (625, 679) 1360054984 -3.8990558046348456 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696778_0.png resize: (485, 716) 1360054985 -4.449263806481577 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696780_0.png resize: (285, 296) 1360054986 -2.8214804248480942 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696786_0.png resize: (394, 469) 1360054987 -4.231985755932019 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696782_0.png resize: (389, 407) 1360054988 -4.3687234837230084 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696787_0.png resize: (152, 289) 1360054989 -3.673066546686017 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696792_0.png resize: (515, 357) 1360054990 -1.8312579769020225 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696795_0.png resize: (136, 146) 1360054991 -3.3970825858170293 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696793_0.png resize: (327, 311) 1360054992 -1.7445019228776946 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696798_0.png resize: (132, 272) 1360054993 -2.0148470546610353 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696789_0.png resize: (622, 652) 1360054994 -3.8613296809000297 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696791_0.png resize: (423, 600) 1360054995 -4.389083881082577 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696788_0.png resize: (242, 262) 1360054996 -0.6098509902848442 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696790_0.png resize: (393, 343) 1360054997 -4.098180781144971 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696794_0.png resize: (183, 313) 1360054998 -3.787930096437325 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696806_0.png resize: (384, 152) 1360054999 -1.9865130590711801 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696809_0.png resize: (426, 656) 1360055000 -2.761347820033321 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696811_0.png resize: (648, 396) 1360055001 -3.412594017705709 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696802_0.png resize: (380, 305) 1360055002 -1.8503769093831801 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696800_0.png resize: (601, 586) 1360055003 -4.137540331518337 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696801_0.png resize: (411, 680) 1360055004 -1.1918056886733999 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696808_0.png resize: (323, 385) 1360055005 -3.300565907624092 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696807_0.png resize: (165, 180) 1360055006 -2.8079114342826923 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696805_0.png resize: (217, 113) 1360055007 -2.0172976665796964 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696803_0.png resize: (537, 469) 1360055008 -2.0205617325055734 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696823_0.png resize: (1033, 1110) 1360055009 -0.15865037172651572 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696827_0.png resize: (740, 493) 1360055010 -2.8976845485556937 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696815_0.png resize: (703, 349) 1360055011 -2.894426861634478 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696821_0.png resize: (420, 200) 1360055012 -2.256928152676312 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696822_0.png resize: (470, 516) 1360055013 -4.101235384092924 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696819_0.png resize: (473, 449) 1360055014 -1.3303285663709057 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696814_0.png resize: (509, 470) 1360055015 -0.2718732479266757 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696820_0.png resize: (507, 472) 1360055017 -3.278226547167457 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696826_0.png resize: (272, 186) 1360055018 -1.9966665863073434 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696813_0.png resize: (612, 271) 1360055019 -3.145186529235006 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696824_0.png resize: (184, 234) 1360055020 -0.4209039508886159 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696818_0.png resize: (294, 151) 1360055021 -0.8100722564712522 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696816_0.png resize: (473, 406) 1360055022 -3.301980645251346 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696747_0.png resize: (106, 93) 1360055025 -1.6108945790180316 treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696763_0.png resize: (309, 685) 1360055026 -3.4409121727450525 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696767_0.png resize: (207, 180) 1360055027 -4.293501206557389 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696796_0.png resize: (191, 306) 1360055028 -4.4065683386648775 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696804_0.png resize: (439, 642) 1360055029 -2.7140130981871646 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696812_0.png resize: (147, 126) 1360055030 -3.2605242276516524 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696828_0.png resize: (154, 132) 1360055032 -4.378686792362817 treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696745_0.png resize: (493, 710) 1360055043 -3.5111193826739875 treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696774_0.png resize: (422, 369) 1360055044 -4.229952407714146 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696797_0.png resize: (125, 227) 1360055045 -2.9677991053831034 treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696810_0.png resize: (269, 413) 1360055047 -4.426465899719076 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696817_0.png resize: (461, 273) 1360055048 -4.256007477876047 treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696825_0.png resize: (376, 294) 1360055049 -3.957094568395211 treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696781_0.png resize: (465, 470) 1360055073 -2.610263025385568 treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696799_0.png resize: (497, 519) 1360055074 -2.4961455585403503 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 104 time used for this insertion : 0.020403146743774414 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 104 time used for this insertion : 0.043471336364746094 save missing photos in datou_result : time spend for datou_step_exec : 34.18089008331299 time spend to save output : 0.07051992416381836 total time spend for step 6 : 34.25141000747681 step7:brightness Wed May 21 16:25:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e.jpg treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66.jpg treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2.jpg treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f.jpg treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5.jpg treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c.jpg treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741.jpg treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696738_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696734_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696739_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696748_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696742_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696744_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696733_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696732_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696743_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696740_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696736_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696746_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696741_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696737_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696735_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696765_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696764_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696760_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696754_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696752_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696756_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696753_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696750_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696749_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696755_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696758_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696762_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696761_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696759_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696757_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696751_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696769_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696771_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696770_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696772_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696768_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696775_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696773_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696776_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696766_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696784_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696783_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696779_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696785_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696777_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696778_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696780_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696786_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696782_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696787_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696792_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696795_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696793_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696798_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696789_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696791_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696788_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696790_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696794_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696806_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696809_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696811_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696802_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696800_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696801_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696808_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696807_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696805_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696803_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696823_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696827_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696815_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696821_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696822_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696819_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696814_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696820_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696826_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696813_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696824_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696818_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696816_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696747_0.png treat image : temp/1747837229_3380934_1360043323_30e534afaab79493a3ce8bc0def42c66_rle_crop_3807696763_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696767_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696796_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696804_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696812_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696828_0.png treat image : temp/1747837229_3380934_1360043487_01edf74b2d0997c3c7b9fb3b7a67028e_rle_crop_3807696745_0.png treat image : temp/1747837229_3380934_1360043304_d95d44765219bee205ddb129b61e1de2_rle_crop_3807696774_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696797_0.png treat image : temp/1747837229_3380934_1360043244_d0f26391a327709cfc3ec31dffff827c_rle_crop_3807696810_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696817_0.png treat image : temp/1747837229_3380934_1360043131_eb95914eb60aa23c6bb5a6c08bd05741_rle_crop_3807696825_0.png treat image : temp/1747837229_3380934_1360043281_e11fab43b394e5f524553b362b01003f_rle_crop_3807696781_0.png treat image : temp/1747837229_3380934_1360043256_2c4e54e801a7968c04d95d93443662f5_rle_crop_3807696799_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 104 time used for this insertion : 0.030614376068115234 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 104 time used for this insertion : 0.03300309181213379 save missing photos in datou_result : time spend for datou_step_exec : 9.215195417404175 time spend to save output : 0.0725092887878418 total time spend for step 7 : 9.287704706192017 step8:velours_tree Wed May 21 16:25:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.4370250701904297 time spend to save output : 9.1552734375e-05 total time spend for step 8 : 0.4371166229248047 step9:send_mail_cod Wed May 21 16:25:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23213504_21-05-2025_16_25_11.pdf 23214508 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232145081747837511 23214509 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232145091747837513 23214510 change filename to text .change filename to text .imagette232145101747837513 23214511 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232145111747837513 23214512 imagette232145121747837514 23214514 change filename to text .imagette232145141747837514 23214515 imagette232145151747837514 23214516 imagette232145161747837514 23214517 imagette232145171747837514 23214518 imagette232145181747837514 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23213504 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23214508,23214509,23214510,23214511,23214512,23214513,23214514,23214515,23214516,23214517,23214518?tags=papier,pet_clair,autre,carton,background,environnement,metal,pehd,mal_croppe,pet_fonce,flou args[1360043487] : ((1360043487, -5.850898593404707, 492609224), (1360043487, 0.9078084054005637, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com args[1360043323] : ((1360043323, -5.574316014100544, 492609224), (1360043323, 0.5758137617454027, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com args[1360043304] : ((1360043304, -6.604102638031094, 492609224), (1360043304, 1.0322257064899967, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com args[1360043281] : ((1360043281, -6.41228851466266, 492609224), (1360043281, 0.6220344131778358, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com args[1360043256] : ((1360043256, -6.289160162103518, 492609224), (1360043256, 0.6380506324909354, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com args[1360043244] : ((1360043244, -5.600830161529649, 492609224), (1360043244, 0.7588744284716468, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com args[1360043131] : ((1360043131, -5.971320772860409, 492609224), (1360043131, 1.069598463084003, 2107752395), '0.16463150697200177') We are sending mail with results at report@fotonower.com refus_total : 0.16463150697200177 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23213504 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23213504_21-05-2025_16_25_11.pdf results_Auto_P23213504_21-05-2025_16_25_11.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23213504_21-05-2025_16_25_11.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23213504','results_Auto_P23213504_21-05-2025_16_25_11.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23213504_21-05-2025_16_25_11.pdf','pdf','','0.66','0.16463150697200177') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23213504

https://www.fotonower.com/image?json=false&list_photos_id=1360043487
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360043323
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360043304
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360043281
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360043256
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360043244
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360043131
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 16.46%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/23214508?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23214509?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23214510?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23214511?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23214514?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23213504_21-05-2025_16_25_11.pdf.

Lien vers velours :https://www.fotonower.com/velours/23214508,23214509,23214510,23214511,23214512,23214513,23214514,23214515,23214516,23214517,23214518?tags=papier,pet_clair,autre,carton,background,environnement,metal,pehd,mal_croppe,pet_fonce,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 21 May 2025 14:25:17 GMT Content-Length: 0 Connection: close X-Message-Id: pSJe52LYT8-hOgdwfs4tbg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1360043487, 1360043323, 1360043304, 1360043281, 1360043256, 1360043244, 1360043131] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043487', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043323', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043304', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043281', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043256', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043244', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043131', None, None, None, None, None, '2925333') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.015279054641723633 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.6330482959747314 time spend to save output : 0.015510797500610352 total time spend for step 9 : 6.648559093475342 step10:split_time_score Wed May 21 16:25:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 2), ('15', 5)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 21052025 23213504 Nombre de photos uploadées : 7 / 23040 (0%) 21052025 23213504 Nombre de photos taguées (types de déchets): 0 / 7 (0%) 21052025 23213504 Nombre de photos taguées (volume) : 0 / 7 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 7.152557373046875e-06 ??????? elapsed_time : fill_and_build_computed_from_old_data 0.00045609474182128906 elapsed_time : insert_dashboard_record_day_entry 0.0311734676361084 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.1958290412808642 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23198775_21-05-2025_09_15_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23198775 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23198775 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23206862 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23213494 order by id desc limit 1 Qualite : 0.16463150697200177 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23213504_21-05-2025_16_25_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23213504 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23213504 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23213510 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23213515 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23213535 order by id desc limit 1 Qualite : 0.18019374517746914 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23213550_21-05-2025_16_14_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23213550 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23213550 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'21052025': {'nb_upload': 7, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1360043487, 1360043323, 1360043304, 1360043281, 1360043256, 1360043244, 1360043131] Looping around the photos to save general results len do output : 1 /23213504Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043487', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043323', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043304', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043281', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043256', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043244', None, None, None, None, None, '2925333') ('3318', None, None, None, None, None, None, None, '2925333') ('3318', '23213504', '1360043131', None, None, None, None, None, '2925333') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.013735532760620117 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.940019845962524 time spend to save output : 0.014296293258666992 total time spend for step 10 : 4.954316139221191 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 7 set_done_treatment 139.59user 114.84system 4:57.46elapsed 85%CPU (0avgtext+0avgdata 4218116maxresident)k 744744inputs+150024outputs (14350major+10636722minor)pagefaults 0swaps