python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2989571 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2721391'] with mtr_portfolio_ids : ['22027923'] and first list_photo_ids : [] new path : /proc/2989571/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.1355230808258057 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Apr 15 12:20:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3716 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-04-15 12:20:33.189928: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-04-15 12:20:33.263124: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-04-15 12:20:33.265458: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f348c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-04-15 12:20:33.265523: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-04-15 12:20:33.272038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-04-15 12:20:33.554631: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2a327000 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-04-15 12:20:33.554708: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-04-15 12:20:33.588637: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-15 12:20:33.589223: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-15 12:20:33.592168: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-15 12:20:33.655284: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-15 12:20:33.655966: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-15 12:20:33.693459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-15 12:20:33.697555: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-15 12:20:33.737321: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-15 12:20:33.738628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-15 12:20:33.738730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-15 12:20:33.739484: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-15 12:20:33.739504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-15 12:20:33.739515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-15 12:20:33.740912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3264 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-04-15 12:20:34.167270: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-15 12:20:34.167391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-15 12:20:34.167420: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-15 12:20:34.167446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-15 12:20:34.167470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-15 12:20:34.167495: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-15 12:20:34.167519: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-15 12:20:34.167546: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-15 12:20:34.168746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-15 12:20:34.170001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-04-15 12:20:34.170061: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-04-15 12:20:34.170086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-15 12:20:34.170106: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-04-15 12:20:34.170125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-04-15 12:20:34.170145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-04-15 12:20:34.170163: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-04-15 12:20:34.170183: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-15 12:20:34.171252: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-04-15 12:20:34.171286: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-04-15 12:20:34.171297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-04-15 12:20:34.171307: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-04-15 12:20:34.172409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3264 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-04-15 12:20:48.086576: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-04-15 12:20:48.094726: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED 2025-04-15 12:20:48.098298: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED 2025-04-15 12:20:48.108664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-04-15 12:20:48.128869: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-04-15 12:20:48.134570: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-04-15 12:20:53.334199: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.066977: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-04-15 12:20:54.124673: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.124754: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.125373: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.125390: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.132133: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.132157: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.138075: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.138095: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.138670: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.138685: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.164233: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.164297: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.164313: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-04-15 12:20:54.165357: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.165379: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.166419: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.166439: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.173709: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.173735: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-04-15 12:20:54.174740: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.175790: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.176802: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.185377: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.186418: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.201405: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.202127: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.202773: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.203400: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.207470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.208097: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.208711: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.209318: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.210319: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.220467: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.221082: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.230088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.230706: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.231368: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.231984: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.232604: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-04-15 12:20:54.233215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2887647232 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 10.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 error in detect the image : temp/1744712428_2989571_1349859626_d46262d8089dd56a78e46b5d6d54dbc0.jpg 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] [[mrcnn_detection/map/while/body/_1/ToFloat/_80]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] 0 successful operations. 0 derived errors ignored. [Op:__inference_keras_scratch_graph_13584] Function call stack: keras_scratch_graph -> keras_scratch_graph NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 46 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 53 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 Detection mask done ! Trying to reset tf kernel 2990094 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1398 tf kernel not reseted sub process len(results) : 7 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 7 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1257 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0010209083557128906 nb_pixel_total : 37500 time to create 1 rle with old method : 0.0458674430847168 length of segment : 220 time Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 91 chid ids of type : 3594 Number RLEs to save : 30465 save missing photos in datou_result : time spend for datou_step_exec : 68.68718576431274 time spend to save output : 1.781301498413086 total time spend for step 1 : 70.46848726272583 step2:crop_condition Tue Apr 15 12:21:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 91 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 48 About to insert : list_path_to_insert length 48 new photo from crops ! About to upload 48 photos upload in portfolio : 3736932 init cache_photo without model_param we have 48 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744712516_2989571 we have uploaded 48 photos in the portfolio 3736932 time of upload the photos Elapsed time : 13.010820865631104 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 3736932 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744712535_2989571 we have uploaded 10 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.974920988082886 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! About to upload 33 photos upload in portfolio : 3736932 init cache_photo without model_param we have 33 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1744712554_2989571 we have uploaded 33 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.952372312545776 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1349859626, 1349859446, 1349859423, 1349859415, 1349859374, 1349859351, 1349859269, 1349859226] Looping around the photos to save general results len do output : 91 /1349869158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1349869401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859626', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859446', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859423', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859415', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859374', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859351', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859269', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859226', None, None, None, None, None, '2721391') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 281 time used for this insertion : 0.042420387268066406 save_final save missing photos in datou_result : time spend for datou_step_exec : 62.21449589729309 time spend to save output : 0.04508161544799805 total time spend for step 2 : 62.25957751274109 step3:rle_unique_nms_with_priority Tue Apr 15 12:22:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 91 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ No data in photo_id : 1349859626 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 1.6872081756591797 time for calcul the mask position with numpy : 1.0882744789123535 nb_pixel_total : 6977961 time to create 1 rle with new method : 1.2930591106414795 time for calcul the mask position with numpy : 0.023233890533447266 nb_pixel_total : 34779 time to create 1 rle with old method : 0.0419766902923584 time for calcul the mask position with numpy : 0.025176525115966797 nb_pixel_total : 37500 time to create 1 rle with old method : 0.04942202568054199 create new chi : 2.561173677444458 time to delete rle : 0.029361248016357422 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2982 TO DO : save crop sub photo not yet done ! save time : 0.2082505226135254 nb_obj : 21 nb_hashtags : 3 time to prepare the origin masks : 13.27949333190918 time for calcul the mask position with numpy : 0.6162889003753662 nb_pixel_total : 5323239 time to create 1 rle with new method : 1.016453504562378 time for calcul the mask position with numpy : 0.04181647300720215 nb_pixel_total : 41161 time to create 1 rle with old method : 0.04603743553161621 time for calcul the mask position with numpy : 0.03651762008666992 nb_pixel_total : 110206 time to create 1 rle with old method : 0.1368405818939209 time for calcul the mask position with numpy : 0.03954029083251953 nb_pixel_total : 38663 time to create 1 rle with old method : 0.047957658767700195 time for calcul the mask position with numpy : 0.03822731971740723 nb_pixel_total : 57909 time to create 1 rle with old method : 0.06773591041564941 time for calcul the mask position with numpy : 0.035036563873291016 nb_pixel_total : 54008 time to create 1 rle with old method : 0.062071800231933594 time for calcul the mask position with numpy : 0.03752779960632324 nb_pixel_total : 106395 time to create 1 rle with old method : 0.12894392013549805 time for calcul the mask position with numpy : 0.046064138412475586 nb_pixel_total : 171508 time to create 1 rle with new method : 0.9472947120666504 time for calcul the mask position with numpy : 0.0359196662902832 nb_pixel_total : 120133 time to create 1 rle with old method : 0.16063308715820312 time for calcul the mask position with numpy : 0.03707265853881836 nb_pixel_total : 156661 time to create 1 rle with new method : 2.5256426334381104 time for calcul the mask position with numpy : 0.03732776641845703 nb_pixel_total : 27792 time to create 1 rle with old method : 0.031139850616455078 time for calcul the mask position with numpy : 0.03633713722229004 nb_pixel_total : 69221 time to create 1 rle with old method : 0.07717680931091309 time for calcul the mask position with numpy : 0.03832244873046875 nb_pixel_total : 42113 time to create 1 rle with old method : 0.04710841178894043 time for calcul the mask position with numpy : 0.038116455078125 nb_pixel_total : 59809 time to create 1 rle with old method : 0.0679771900177002 time for calcul the mask position with numpy : 0.03629755973815918 nb_pixel_total : 51766 time to create 1 rle with old method : 0.058135032653808594 time for calcul the mask position with numpy : 0.036133766174316406 nb_pixel_total : 31067 time to create 1 rle with old method : 0.03465390205383301 time for calcul the mask position with numpy : 0.03617215156555176 nb_pixel_total : 121686 time to create 1 rle with old method : 0.13717126846313477 time for calcul the mask position with numpy : 0.035272836685180664 nb_pixel_total : 95264 time to create 1 rle with old method : 0.12553739547729492 time for calcul the mask position with numpy : 0.03689432144165039 nb_pixel_total : 37593 time to create 1 rle with old method : 0.06544780731201172 time for calcul the mask position with numpy : 0.039931297302246094 nb_pixel_total : 61622 time to create 1 rle with old method : 0.07468843460083008 time for calcul the mask position with numpy : 0.037046194076538086 nb_pixel_total : 162097 time to create 1 rle with new method : 0.7480316162109375 time for calcul the mask position with numpy : 0.04105186462402344 nb_pixel_total : 110327 time to create 1 rle with old method : 0.1256566047668457 create new chi : 8.264974117279053 time to delete rle : 0.0032808780670166016 batch 1 Loaded 43 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 17115 TO DO : save crop sub photo not yet done ! save time : 0.9905447959899902 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 4.165484666824341 time for calcul the mask position with numpy : 0.9154300689697266 nb_pixel_total : 6327474 time to create 1 rle with new method : 1.1648521423339844 time for calcul the mask position with numpy : 0.02435016632080078 nb_pixel_total : 4039 time to create 1 rle with old method : 0.006275177001953125 time for calcul the mask position with numpy : 0.02143716812133789 nb_pixel_total : 37138 time to create 1 rle with old method : 0.041925907135009766 time for calcul the mask position with numpy : 0.022469758987426758 nb_pixel_total : 81725 time to create 1 rle with old method : 0.11417126655578613 time for calcul the mask position with numpy : 0.023015975952148438 nb_pixel_total : 117142 time to create 1 rle with old method : 0.13073492050170898 time for calcul the mask position with numpy : 0.021495580673217773 nb_pixel_total : 67280 time to create 1 rle with old method : 0.07509970664978027 time for calcul the mask position with numpy : 0.022302865982055664 nb_pixel_total : 158181 time to create 1 rle with new method : 0.5028083324432373 time for calcul the mask position with numpy : 0.03613781929016113 nb_pixel_total : 114126 time to create 1 rle with old method : 0.13296151161193848 time for calcul the mask position with numpy : 0.03516864776611328 nb_pixel_total : 70906 time to create 1 rle with old method : 0.07893252372741699 time for calcul the mask position with numpy : 0.033658504486083984 nb_pixel_total : 72229 time to create 1 rle with old method : 0.08017754554748535 create new chi : 3.554717540740967 time to delete rle : 0.001247406005859375 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++Number RLEs to save : 7796 TO DO : save crop sub photo not yet done ! save time : 0.50612473487854 nb_obj : 13 nb_hashtags : 2 time to prepare the origin masks : 8.113306760787964 time for calcul the mask position with numpy : 0.754568338394165 nb_pixel_total : 6515665 time to create 1 rle with new method : 0.7429521083831787 time for calcul the mask position with numpy : 0.027410030364990234 nb_pixel_total : 22703 time to create 1 rle with old method : 0.025445222854614258 time for calcul the mask position with numpy : 0.025661468505859375 nb_pixel_total : 147790 time to create 1 rle with old method : 0.16771841049194336 time for calcul the mask position with numpy : 0.025330066680908203 nb_pixel_total : 27326 time to create 1 rle with old method : 0.03427696228027344 time for calcul the mask position with numpy : 0.02348470687866211 nb_pixel_total : 35972 time to create 1 rle with old method : 0.041778564453125 time for calcul the mask position with numpy : 0.023091554641723633 nb_pixel_total : 12771 time to create 1 rle with old method : 0.014816761016845703 time for calcul the mask position with numpy : 0.0250701904296875 nb_pixel_total : 3676 time to create 1 rle with old method : 0.0045375823974609375 time for calcul the mask position with numpy : 0.03326916694641113 nb_pixel_total : 23465 time to create 1 rle with old method : 0.03679251670837402 time for calcul the mask position with numpy : 0.022246360778808594 nb_pixel_total : 37396 time to create 1 rle with old method : 0.04256272315979004 time for calcul the mask position with numpy : 0.02174663543701172 nb_pixel_total : 11664 time to create 1 rle with old method : 0.01322793960571289 time for calcul the mask position with numpy : 0.024854660034179688 nb_pixel_total : 63124 time to create 1 rle with old method : 0.092559814453125 time for calcul the mask position with numpy : 0.03810381889343262 nb_pixel_total : 94400 time to create 1 rle with old method : 0.10743331909179688 time for calcul the mask position with numpy : 0.03303241729736328 nb_pixel_total : 20011 time to create 1 rle with old method : 0.022809505462646484 time for calcul the mask position with numpy : 0.022202253341674805 nb_pixel_total : 34277 time to create 1 rle with old method : 0.03877067565917969 create new chi : 2.5264198780059814 time to delete rle : 0.0014214515686035156 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 9192 TO DO : save crop sub photo not yet done ! save time : 0.5596737861633301 nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 4.570672988891602 time for calcul the mask position with numpy : 1.4649739265441895 nb_pixel_total : 6537160 time to create 1 rle with new method : 1.105367660522461 time for calcul the mask position with numpy : 0.022736549377441406 nb_pixel_total : 7157 time to create 1 rle with old method : 0.01163935661315918 time for calcul the mask position with numpy : 0.023178577423095703 nb_pixel_total : 159052 time to create 1 rle with new method : 1.1391191482543945 time for calcul the mask position with numpy : 0.03596663475036621 nb_pixel_total : 64707 time to create 1 rle with old method : 0.07626867294311523 time for calcul the mask position with numpy : 0.03601527214050293 nb_pixel_total : 59675 time to create 1 rle with old method : 0.0664069652557373 time for calcul the mask position with numpy : 0.02217721939086914 nb_pixel_total : 37158 time to create 1 rle with old method : 0.04144644737243652 time for calcul the mask position with numpy : 0.02117633819580078 nb_pixel_total : 34151 time to create 1 rle with old method : 0.03935670852661133 time for calcul the mask position with numpy : 0.022365808486938477 nb_pixel_total : 43149 time to create 1 rle with old method : 0.04886507987976074 time for calcul the mask position with numpy : 0.022663593292236328 nb_pixel_total : 108031 time to create 1 rle with old method : 0.12352490425109863 create new chi : 4.40057635307312 time to delete rle : 0.0011186599731445312 batch 1 Loaded 17 chid ids of type : 3594 ++++++++Number RLEs to save : 7676 TO DO : save crop sub photo not yet done ! save time : 0.4647867679595947 nb_obj : 27 nb_hashtags : 3 time to prepare the origin masks : 4.623351812362671 time for calcul the mask position with numpy : 1.5457079410552979 nb_pixel_total : 5287653 time to create 1 rle with new method : 1.8671960830688477 time for calcul the mask position with numpy : 0.03192496299743652 nb_pixel_total : 22853 time to create 1 rle with old method : 0.029009580612182617 time for calcul the mask position with numpy : 0.03156542778015137 nb_pixel_total : 59359 time to create 1 rle with old method : 0.0680396556854248 time for calcul the mask position with numpy : 0.02910470962524414 nb_pixel_total : 22277 time to create 1 rle with old method : 0.024964570999145508 time for calcul the mask position with numpy : 0.029898881912231445 nb_pixel_total : 38349 time to create 1 rle with old method : 0.04353809356689453 time for calcul the mask position with numpy : 0.03650641441345215 nb_pixel_total : 39640 time to create 1 rle with old method : 0.05900239944458008 time for calcul the mask position with numpy : 0.03609943389892578 nb_pixel_total : 162119 time to create 1 rle with new method : 0.6286942958831787 time for calcul the mask position with numpy : 0.030295610427856445 nb_pixel_total : 231514 time to create 1 rle with new method : 0.9302980899810791 time for calcul the mask position with numpy : 0.029387712478637695 nb_pixel_total : 16717 time to create 1 rle with old method : 0.020135879516601562 time for calcul the mask position with numpy : 0.03075695037841797 nb_pixel_total : 36614 time to create 1 rle with old method : 0.044078826904296875 time for calcul the mask position with numpy : 0.03333473205566406 nb_pixel_total : 69369 time to create 1 rle with old method : 0.0903322696685791 time for calcul the mask position with numpy : 0.03054499626159668 nb_pixel_total : 52920 time to create 1 rle with old method : 0.06013917922973633 time for calcul the mask position with numpy : 0.029405593872070312 nb_pixel_total : 1401 time to create 1 rle with old method : 0.00185394287109375 time for calcul the mask position with numpy : 0.029854774475097656 nb_pixel_total : 20267 time to create 1 rle with old method : 0.023195266723632812 time for calcul the mask position with numpy : 0.032585859298706055 nb_pixel_total : 47172 time to create 1 rle with old method : 0.055793046951293945 time for calcul the mask position with numpy : 0.031048059463500977 nb_pixel_total : 46977 time to create 1 rle with old method : 0.05889320373535156 time for calcul the mask position with numpy : 0.03151965141296387 nb_pixel_total : 100801 time to create 1 rle with old method : 0.11411809921264648 time for calcul the mask position with numpy : 0.03017735481262207 nb_pixel_total : 63565 time to create 1 rle with old method : 0.07155942916870117 time for calcul the mask position with numpy : 0.029863357543945312 nb_pixel_total : 26456 time to create 1 rle with old method : 0.030034303665161133 time for calcul the mask position with numpy : 0.03028559684753418 nb_pixel_total : 71852 time to create 1 rle with old method : 0.08359241485595703 time for calcul the mask position with numpy : 0.030094385147094727 nb_pixel_total : 37730 time to create 1 rle with old method : 0.042331695556640625 time for calcul the mask position with numpy : 0.030191659927368164 nb_pixel_total : 94752 time to create 1 rle with old method : 0.1081092357635498 time for calcul the mask position with numpy : 0.03283047676086426 nb_pixel_total : 182845 time to create 1 rle with new method : 1.1151623725891113 time for calcul the mask position with numpy : 0.032738685607910156 nb_pixel_total : 68453 time to create 1 rle with old method : 0.07726764678955078 time for calcul the mask position with numpy : 0.030825376510620117 nb_pixel_total : 83344 time to create 1 rle with old method : 0.09258246421813965 time for calcul the mask position with numpy : 0.030092239379882812 nb_pixel_total : 132070 time to create 1 rle with old method : 0.14797306060791016 time for calcul the mask position with numpy : 0.030698537826538086 nb_pixel_total : 27903 time to create 1 rle with old method : 0.04466414451599121 time for calcul the mask position with numpy : 0.032510995864868164 nb_pixel_total : 5268 time to create 1 rle with old method : 0.0060405731201171875 create new chi : 8.453755378723145 time to delete rle : 0.002395153045654297 batch 1 Loaded 55 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++Number RLEs to save : 19446 TO DO : save crop sub photo not yet done ! save time : 1.1764249801635742 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 7.0503621101379395 time for calcul the mask position with numpy : 1.1171092987060547 nb_pixel_total : 6182109 time to create 1 rle with new method : 1.2710726261138916 time for calcul the mask position with numpy : 0.042116641998291016 nb_pixel_total : 35286 time to create 1 rle with old method : 0.052982330322265625 time for calcul the mask position with numpy : 0.04163098335266113 nb_pixel_total : 56239 time to create 1 rle with old method : 0.06389760971069336 time for calcul the mask position with numpy : 0.035837411880493164 nb_pixel_total : 70219 time to create 1 rle with old method : 0.07894563674926758 time for calcul the mask position with numpy : 0.03587055206298828 nb_pixel_total : 143231 time to create 1 rle with old method : 0.16023778915405273 time for calcul the mask position with numpy : 0.036217451095581055 nb_pixel_total : 230974 time to create 1 rle with new method : 2.058110237121582 time for calcul the mask position with numpy : 0.06116819381713867 nb_pixel_total : 107405 time to create 1 rle with old method : 0.16860723495483398 time for calcul the mask position with numpy : 0.037520408630371094 nb_pixel_total : 41009 time to create 1 rle with old method : 0.06003975868225098 time for calcul the mask position with numpy : 0.0415956974029541 nb_pixel_total : 65516 time to create 1 rle with old method : 0.07321310043334961 time for calcul the mask position with numpy : 0.0383601188659668 nb_pixel_total : 8535 time to create 1 rle with old method : 0.009609699249267578 time for calcul the mask position with numpy : 0.03941822052001953 nb_pixel_total : 68520 time to create 1 rle with old method : 0.08317041397094727 time for calcul the mask position with numpy : 0.0387423038482666 nb_pixel_total : 41197 time to create 1 rle with old method : 0.047152042388916016 create new chi : 5.761300086975098 time to delete rle : 0.0020058155059814453 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 10914 TO DO : save crop sub photo not yet done ! save time : 0.6625914573669434 map_output_result : {1349859626: (0.0, 'Should be the crop_list due to order', 0.0), 1349859446: (0.0, 'Should be the crop_list due to order', 0), 1349859423: (0.0, 'Should be the crop_list due to order', 0), 1349859415: (0.0, 'Should be the crop_list due to order', 0), 1349859374: (0.0, 'Should be the crop_list due to order', 0), 1349859351: (0.0, 'Should be the crop_list due to order', 0), 1349859269: (0.0, 'Should be the crop_list due to order', 0), 1349859226: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1349859626, 1349859446, 1349859423, 1349859415, 1349859374, 1349859351, 1349859269, 1349859226] Looping around the photos to save general results len do output : 8 /1349859626.Didn't retrieve data . /1349859446.Didn't retrieve data . /1349859423.Didn't retrieve data . /1349859415.Didn't retrieve data . /1349859374.Didn't retrieve data . /1349859351.Didn't retrieve data . /1349859269.Didn't retrieve data . /1349859226.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859626', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859446', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859423', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859415', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859374', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859351', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859269', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859226', None, None, None, None, None, '2721391') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.012630462646484375 save_final save missing photos in datou_result : time spend for datou_step_exec : 85.02187919616699 time spend to save output : 0.013318777084350586 total time spend for step 3 : 85.03519797325134 step4:ventilate_hashtags_in_portfolio Tue Apr 15 12:24:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22027923 get user id for portfolio 22027923 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22027923 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','metal','pet_fonce','mal_croppe','background','carton','pehd','flou','pet_clair','papier','environnement')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22027923 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','metal','pet_fonce','mal_croppe','background','carton','pehd','flou','pet_clair','papier','environnement')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22027923 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','metal','pet_fonce','mal_croppe','background','carton','pehd','flou','pet_clair','papier','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/22027996,22027997,22027998,22027999,22028000,22028001,22028002,22028003,22028004,22028005,22028006?tags=autre,metal,pet_fonce,mal_croppe,background,carton,pehd,flou,pet_clair,papier,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1349859626, 1349859446, 1349859423, 1349859415, 1349859374, 1349859351, 1349859269, 1349859226] Looping around the photos to save general results len do output : 1 /22027923. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859626', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859446', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859423', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859415', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859374', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859351', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859269', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859226', None, None, None, None, None, '2721391') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.012438058853149414 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5120513439178467 time spend to save output : 0.012798547744750977 total time spend for step 4 : 1.5248498916625977 step5:final Tue Apr 15 12:24:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1349859626: ('0.10993276470020877',), 1349859446: ('0.10993276470020877',), 1349859423: ('0.10993276470020877',), 1349859415: ('0.10993276470020877',), 1349859374: ('0.10993276470020877',), 1349859351: ('0.10993276470020877',), 1349859269: ('0.10993276470020877',), 1349859226: ('0.10993276470020877',)} new output for save of step final : {1349859626: ('0.10993276470020877',), 1349859446: ('0.10993276470020877',), 1349859423: ('0.10993276470020877',), 1349859415: ('0.10993276470020877',), 1349859374: ('0.10993276470020877',), 1349859351: ('0.10993276470020877',), 1349859269: ('0.10993276470020877',), 1349859226: ('0.10993276470020877',)} [1349859626, 1349859446, 1349859423, 1349859415, 1349859374, 1349859351, 1349859269, 1349859226] Looping around the photos to save general results len do output : 8 /1349859626.Didn't retrieve data . /1349859446.Didn't retrieve data . /1349859423.Didn't retrieve data . /1349859415.Didn't retrieve data . /1349859374.Didn't retrieve data . /1349859351.Didn't retrieve data . /1349859269.Didn't retrieve data . /1349859226.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859626', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859446', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859423', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859415', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859374', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859351', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859269', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859226', None, None, None, None, None, '2721391') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.015273809432983398 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10236883163452148 time spend to save output : 0.015803098678588867 total time spend for step 5 : 0.11817193031311035 step6:blur_detection Tue Apr 15 12:24:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1744712428_2989571_1349859626_d46262d8089dd56a78e46b5d6d54dbc0.jpg resize: (2160, 3264) 1349859626 -0.614122050820776 treat image : temp/1744712428_2989571_1349859446_53dfad24fddfa6559d9ab624d0062820.jpg resize: (2160, 3264) 1349859446 -2.2954933487570535 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a.jpg resize: (2160, 3264) 1349859423 -2.600720073326245 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493.jpg resize: (2160, 3264) 1349859415 -0.9545731760582741 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35.jpg resize: (2160, 3264) 1349859374 0.34091412000240556 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35.jpg resize: (2160, 3264) 1349859351 -0.9503460712127892 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545.jpg resize: (2160, 3264) 1349859269 -3.654009251701781 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d.jpg resize: (2160, 3264) 1349859226 -0.6857051354454295 treat image : temp/1744712428_2989571_1349859446_53dfad24fddfa6559d9ab624d0062820_rle_crop_3746522722_0.png resize: (174, 315) 1349869158 -2.8389456164208138 treat image : temp/1744712428_2989571_1349859446_53dfad24fddfa6559d9ab624d0062820_rle_crop_3746522723_0.png resize: (191, 214) 1349869159 -1.4341671980215822 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522728_0.png resize: (343, 465) 1349869160 -1.8665032269355613 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522730_0.png resize: (235, 220) 1349869161 -1.8556468326037086 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522734_0.png resize: (404, 312) 1349869162 -1.5806071738777794 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522725_0.png resize: (529, 452) 1349869163 -0.19477614789451458 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522736_0.png resize: (416, 533) 1349869164 -1.7681740462253535 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522740_0.png resize: (236, 318) 1349869165 -2.159094213816078 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522731_0.png resize: (220, 385) 1349869167 -1.4990514141664122 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522726_0.png resize: (369, 224) 1349869168 -1.2884059328307336 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522742_0.png resize: (303, 206) 1349869169 -0.08225120011368506 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522745_0.png resize: (199, 568) 1349869170 -0.9603159458646048 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522750_0.png resize: (450, 427) 1349869172 -2.1736455474878205 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522753_0.png resize: (60, 92) 1349869174 0.4439931140389538 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522761_0.png resize: (247, 266) 1349869176 -1.4977682496326452 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522755_0.png resize: (177, 132) 1349869178 -0.09802324438940498 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522756_0.png resize: (352, 424) 1349869180 0.9860390092848481 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522759_0.png resize: (364, 145) 1349869182 -0.43177494422160073 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522762_0.png resize: (114, 145) 1349869184 0.3127098560569044 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522754_0.png resize: (331, 254) 1349869186 -1.3816783022102785 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522767_0.png resize: (483, 247) 1349869188 0.6453915697027238 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522772_0.png resize: (355, 234) 1349869191 -0.40065819540578035 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522774_0.png resize: (118, 76) 1349869192 0.646669492341985 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522773_0.png resize: (666, 391) 1349869193 -2.141896649804289 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522768_0.png resize: (385, 381) 1349869194 -1.2443580249103707 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522777_0.png resize: (426, 322) 1349869195 -0.999196926351283 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522785_0.png resize: (444, 121) 1349869196 -0.34559854507766474 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522780_0.png resize: (469, 204) 1349869197 -1.217235773247131 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522794_0.png resize: (183, 262) 1349869198 -2.4329791721743272 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522797_0.png resize: (234, 241) 1349869199 -2.238013260381267 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522779_0.png resize: (219, 229) 1349869200 -2.128512503576491 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522778_0.png resize: (365, 477) 1349869201 -1.1208364364138526 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522790_0.png resize: (278, 117) 1349869202 -2.3763973147603332 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522800_0.png resize: (332, 398) 1349869203 -2.519257367288205 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522776_0.png resize: (442, 359) 1349869204 -2.595074835495337 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522775_0.png resize: (356, 469) 1349869205 -1.8522872666921155 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522784_0.png resize: (515, 515) 1349869206 -1.903096816264097 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522798_0.png resize: (87, 138) 1349869207 -2.870185547871363 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522781_0.png resize: (504, 365) 1349869208 -3.179985373618402 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522786_0.png resize: (242, 323) 1349869209 -2.485871016817658 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522801_0.png resize: (358, 355) 1349869210 -1.8702205477388194 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522802_0.png resize: (228, 271) 1349869211 -1.0236226613329438 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522803_0.png resize: (402, 226) 1349869212 -0.4288873430240614 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522804_0.png resize: (82, 136) 1349869213 -0.3725878011844139 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522807_0.png resize: (436, 403) 1349869214 0.49874646643611115 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522812_0.png resize: (245, 587) 1349869215 -1.4226711273756492 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522805_0.png resize: (480, 190) 1349869217 -1.4177761848273607 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522810_0.png resize: (296, 302) 1349869218 -2.551855543348092 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522724_0.png resize: (348, 457) 1349869234 -1.5268328648790765 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522732_0.png resize: (222, 334) 1349869235 -1.905596612682703 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522733_0.png resize: (347, 230) 1349869236 -2.2327874147900686 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522771_0.png resize: (385, 351) 1349869237 -0.39307591201954023 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522769_0.png resize: (192, 213) 1349869238 1.3041614084508486 treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522770_0.png resize: (237, 347) 1349869239 -1.7931337976876447 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522799_0.png resize: (186, 261) 1349869240 -2.332970398985531 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522791_0.png resize: (303, 380) 1349869241 -2.972708969753351 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522782_0.png resize: (159, 471) 1349869242 -3.0192747382346465 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522788_0.png resize: (210, 229) 1349869243 -3.6973007128024795 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522743_0.png resize: (398, 420) 1349869367 -2.1981671951885824 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522741_0.png resize: (375, 181) 1349869368 -1.8162567530434874 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522727_0.png resize: (165, 279) 1349869369 -2.076599334679893 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522737_0.png resize: (399, 440) 1349869370 -1.1682317856221371 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522739_0.png resize: (371, 351) 1349869371 0.15173618979355036 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522729_0.png resize: (266, 616) 1349869372 -3.180560131018535 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522744_0.png resize: (285, 233) 1349869373 -3.073066111111053 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522738_0.png resize: (526, 542) 1349869374 -1.8528481703571638 treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522735_0.png resize: (236, 170) 1349869375 -2.293655773887676 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522748_0.png resize: (562, 407) 1349869376 0.13272441694657852 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522746_0.png resize: (194, 503) 1349869377 -1.9938904816021634 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522749_0.png resize: (303, 301) 1349869378 -0.9574940542336121 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522747_0.png resize: (349, 446) 1349869379 -1.5416353674341376 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522752_0.png resize: (209, 225) 1349869380 -0.8676152308729784 treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522751_0.png resize: (394, 341) 1349869382 -1.2949218046441675 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522757_0.png resize: (331, 334) 1349869383 -1.3092371363530686 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522760_0.png resize: (231, 115) 1349869384 0.751493779829821 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522766_0.png resize: (180, 168) 1349869385 -1.8711977722672168 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522763_0.png resize: (231, 288) 1349869386 -1.6247275689407061 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522765_0.png resize: (382, 447) 1349869387 -2.6142126933785454 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522758_0.png resize: (128, 152) 1349869389 -1.0192937084636493 treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522764_0.png resize: (310, 188) 1349869390 -1.2298715261908988 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522796_0.png resize: (141, 158) 1349869391 -2.3855844641509014 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522795_0.png resize: (205, 307) 1349869392 -1.9568114045224743 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522793_0.png resize: (208, 248) 1349869393 -2.2200833934132587 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522792_0.png resize: (332, 151) 1349869394 -1.557516651423761 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522787_0.png resize: (155, 200) 1349869395 -3.8365000264902447 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522783_0.png resize: (364, 206) 1349869396 -4.255959188218053 treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522789_0.png resize: (447, 671) 1349869397 -2.7702088463204944 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522808_0.png resize: (564, 685) 1349869398 -1.005890614358648 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522806_0.png resize: (210, 310) 1349869399 -1.956982706428872 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522809_0.png resize: (655, 253) 1349869400 -1.5870756339296137 treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522811_0.png resize: (352, 170) 1349869401 -1.7935754847569247 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 99 time used for this insertion : 0.017233610153198242 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 99 time used for this insertion : 0.016113996505737305 save missing photos in datou_result : time spend for datou_step_exec : 38.47599983215332 time spend to save output : 0.03845858573913574 total time spend for step 6 : 38.514458417892456 step7:brightness Tue Apr 15 12:24:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1744712428_2989571_1349859626_d46262d8089dd56a78e46b5d6d54dbc0.jpg treat image : temp/1744712428_2989571_1349859446_53dfad24fddfa6559d9ab624d0062820.jpg treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a.jpg treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493.jpg treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35.jpg treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35.jpg treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545.jpg treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d.jpg treat image : temp/1744712428_2989571_1349859446_53dfad24fddfa6559d9ab624d0062820_rle_crop_3746522722_0.png treat image : temp/1744712428_2989571_1349859446_53dfad24fddfa6559d9ab624d0062820_rle_crop_3746522723_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522728_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522730_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522734_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522725_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522736_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522740_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522731_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522726_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522742_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522745_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522750_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522753_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522761_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522755_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522756_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522759_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522762_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522754_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522767_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522772_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522774_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522773_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522768_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522777_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522785_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522780_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522794_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522797_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522779_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522778_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522790_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522800_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522776_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522775_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522784_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522798_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522781_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522786_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522801_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522802_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522803_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522804_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522807_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522812_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522805_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522810_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522724_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522732_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522733_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522771_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522769_0.png treat image : temp/1744712428_2989571_1349859351_1693366a4beaddf3b413a175dab60f35_rle_crop_3746522770_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522799_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522791_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522782_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522788_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522743_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522741_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522727_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522737_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522739_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522729_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522744_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522738_0.png treat image : temp/1744712428_2989571_1349859423_96207e9fcea0d4cc67fb27f1392f159a_rle_crop_3746522735_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522748_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522746_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522749_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522747_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522752_0.png treat image : temp/1744712428_2989571_1349859415_7ab48d944e432941a84e08e6f2f39493_rle_crop_3746522751_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522757_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522760_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522766_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522763_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522765_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522758_0.png treat image : temp/1744712428_2989571_1349859374_3fe74a8e3b6589541511f080d7b87d35_rle_crop_3746522764_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522796_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522795_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522793_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522792_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522787_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522783_0.png treat image : temp/1744712428_2989571_1349859269_c00d6f3fc974b51d38d5a10c2588f545_rle_crop_3746522789_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522808_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522806_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522809_0.png treat image : temp/1744712428_2989571_1349859226_898da403dbc9274badd959fc1082d19d_rle_crop_3746522811_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 99 time used for this insertion : 0.01825714111328125 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 99 time used for this insertion : 0.015353679656982422 save missing photos in datou_result : time spend for datou_step_exec : 7.7666778564453125 time spend to save output : 0.03868842124938965 total time spend for step 7 : 7.805366277694702 step8:velours_tree Tue Apr 15 12:24:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.20916199684143066 time spend to save output : 4.38690185546875e-05 total time spend for step 8 : 0.20920586585998535 step9:send_mail_cod Tue Apr 15 12:24:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22027923_15-04-2025_12_24_55.pdf 22027996 imagette220279961744712695 22027997 imagette220279971744712695 22027998 imagette220279981744712695 22027999 imagette220279991744712695 22028000 imagette220280001744712695 22028001 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette220280011744712695 22028002 imagette220280021744712695 22028003 imagette220280031744712695 22028004 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette220280041744712695 22028005 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette220280051744712697 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22027923 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22027996,22027997,22027998,22027999,22028000,22028001,22028002,22028003,22028004,22028005,22028006?tags=autre,metal,pet_fonce,mal_croppe,background,carton,pehd,flou,pet_clair,papier,environnement args[1349859626] : ((1349859626, -0.614122050820776, 492688767), (1349859626, 0.11858949751600897, 2107752395), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859446] : ((1349859446, -2.2954933487570535, 492609224), (1349859446, 0.6383391865660117, 2107752395), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859423] : ((1349859423, -2.600720073326245, 492609224), (1349859423, -0.09539470756234009, 496442774), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859415] : ((1349859415, -0.9545731760582741, 492688767), (1349859415, 0.2493452693146176, 2107752395), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859374] : ((1349859374, 0.34091412000240556, 492688767), (1349859374, -0.19491395156210187, 496442774), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859351] : ((1349859351, -0.9503460712127892, 492688767), (1349859351, 0.43883980621693314, 2107752395), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859269] : ((1349859269, -3.654009251701781, 492609224), (1349859269, -0.023202596261349337, 2107752395), '0.10993276470020877') We are sending mail with results at report@fotonower.com args[1349859226] : ((1349859226, -0.6857051354454295, 492688767), (1349859226, -0.10281665412164785, 496442774), '0.10993276470020877') We are sending mail with results at report@fotonower.com refus_total : 0.10993276470020877 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=22027923 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1349859269,1349859226,1349859351,1349859374,1349859415,1349859423,1349859446,1349859626) Found this number of photos: 8 begin to download photo : 1349859269 begin to download photo : 1349859351 begin to download photo : 1349859415 begin to download photo : 1349859446 download finish for photo 1349859415 begin to download photo : 1349859423 download finish for photo 1349859351 begin to download photo : 1349859374 download finish for photo 1349859446 begin to download photo : 1349859626 download finish for photo 1349859269 begin to download photo : 1349859226 download finish for photo 1349859374 download finish for photo 1349859423 download finish for photo 1349859626 download finish for photo 1349859226 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22027923_15-04-2025_12_24_55.pdf results_Auto_P22027923_15-04-2025_12_24_55.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22027923_15-04-2025_12_24_55.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','22027923','results_Auto_P22027923_15-04-2025_12_24_55.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22027923_15-04-2025_12_24_55.pdf','pdf','','0.58','0.10993276470020877') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/22027923

https://www.fotonower.com/image?json=false&list_photos_id=1349859626
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859446
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859423
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859415
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859374
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859351
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859269
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1349859226
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 10.99%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/22028001?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/22028004?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/22028005?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22027923_15-04-2025_12_24_55.pdf.

Lien vers velours :https://www.fotonower.com/velours/22027996,22027997,22027998,22027999,22028000,22028001,22028002,22028003,22028004,22028005,22028006?tags=autre,metal,pet_fonce,mal_croppe,background,carton,pehd,flou,pet_clair,papier,environnement.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 15 Apr 2025 10:25:01 GMT Content-Length: 0 Connection: close X-Message-Id: DBnMkfvST0yFQ8B5kO5DMw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1349859626, 1349859446, 1349859423, 1349859415, 1349859374, 1349859351, 1349859269, 1349859226] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859626', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859446', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859423', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859415', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859374', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859351', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859269', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859226', None, None, None, None, None, '2721391') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.01357269287109375 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.5450804233551025 time spend to save output : 0.013841390609741211 total time spend for step 9 : 6.558921813964844 step10:split_time_score Tue Apr 15 12:25:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 15042025 22027923 Nombre de photos uploadées : 8 / 23040 (0%) 15042025 22027923 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 15042025 22027923 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 4.5299530029296875e-06 elapsed_time : order_list_meta_photo_and_scores 1.0967254638671875e-05 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0005114078521728516 elapsed_time : insert_dashboard_record_day_entry 0.022509336471557617 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22027920 order by id desc limit 1 Qualite : 0.10993276470020877 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22027923_15-04-2025_12_24_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22027923 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22027923 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'15042025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1349859626, 1349859446, 1349859423, 1349859415, 1349859374, 1349859351, 1349859269, 1349859226] Looping around the photos to save general results len do output : 1 /22027923Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859626', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859446', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859423', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859415', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859374', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859351', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859269', None, None, None, None, None, '2721391') ('3318', None, None, None, None, None, None, None, '2721391') ('3318', '22027923', '1349859226', None, None, None, None, None, '2721391') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013413190841674805 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.0662157535552979 time spend to save output : 0.013686656951904297 total time spend for step 10 : 1.0799024105072021 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 117.82user 122.86system 4:37.43elapsed 86%CPU (0avgtext+0avgdata 4701924maxresident)k 1247688inputs+81176outputs (4358major+9267359minor)pagefaults 0swaps