TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2898167'] with mtr_portfolio_ids : ['23133812'] and first list_photo_ids : [] new path : /proc/3190892/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.7822699546813965 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon May 19 19:40:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10583 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-19 19:40:35.020048: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-19 19:40:35.047165: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-19 19:40:35.049145: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd9a8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-19 19:40:35.049174: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-19 19:40:35.053226: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-19 19:40:35.197791: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x40adadf0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-19 19:40:35.197849: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-19 19:40:35.199121: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 19:40:35.199562: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 19:40:35.202801: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 19:40:35.205955: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 19:40:35.206487: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 19:40:35.209424: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 19:40:35.210555: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 19:40:35.215344: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 19:40:35.216825: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 19:40:35.216905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 19:40:35.217649: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-19 19:40:35.217665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-19 19:40:35.217674: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-19 19:40:35.219033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9527 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-19 19:40:35.504068: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 19:40:35.504269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 19:40:35.504295: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 19:40:35.504317: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 19:40:35.504339: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 19:40:35.504360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 19:40:35.504380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 19:40:35.504401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 19:40:35.506103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 19:40:35.508143: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 19:40:35.508329: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 19:40:35.508355: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 19:40:35.508376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 19:40:35.508396: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 19:40:35.508415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 19:40:35.508434: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 19:40:35.508453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 19:40:35.510156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 19:40:35.510220: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-19 19:40:35.510232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-19 19:40:35.510243: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-19 19:40:35.512060: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9527 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-19 19:40:47.654382: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 19:40:47.861303: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 19:40:49.654613: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.682803: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.683715: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.684576: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.685449: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.62G (2817927680 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.686330: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.36G (2536134912 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.687202: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.12G (2282521344 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.687249: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.688162: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.688181: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.696064: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.696094: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.696980: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.696999: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.703989: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.704037: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.704905: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.704924: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.754005: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.754048: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.754604: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.754621: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.760476: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.760499: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.761048: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.761065: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-19 19:40:49.796045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.796647: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.798518: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.799080: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.843116: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.843691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.845873: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.846417: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.876986: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.877575: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.879340: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.879906: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.886794: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.887387: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.889186: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.889745: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.895717: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.896275: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.897818: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.898374: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.925449: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.926018: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.926562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.927149: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.930640: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.931198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.946662: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.947238: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.947787: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.948331: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.960732: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.961254: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.961751: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.962248: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.966535: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.967110: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.971661: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.972214: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.984306: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.984859: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.988988: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.989540: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.990083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.990626: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.991399: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:49.991949: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.002799: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.003398: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.003960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.004504: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.005047: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.005590: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.006132: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.006673: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.015925: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.016476: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.022672: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.023216: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.049243: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.049322: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-19 19:40:50.050343: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.051382: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.058530: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.059082: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.059642: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.060184: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.068141: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.068696: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.087757: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.088747: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.089603: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.090438: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.094948: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.095840: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.096673: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.097505: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.099414: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.108839: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.109357: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.118343: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.118888: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.119457: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.119991: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.120508: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-19 19:40:50.121015: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 10.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 36 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 35 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 10.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 35 Detection mask done ! Trying to reset tf kernel 3191675 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9179 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10372 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.001373291015625 nb_pixel_total : 28778 time to create 1 rle with old method : 0.034375905990600586 length of segment : 181 time for calcul the mask position with numpy : 0.00212860107421875 nb_pixel_total : 71720 time to create 1 rle with old method : 0.08972716331481934 length of segment : 376 time for calcul the mask position with numpy : 0.002009153366088867 nb_pixel_total : 82518 time to create 1 rle with old method : 0.11029243469238281 length of segment : 635 time for calcul the mask position with numpy : 0.0017249584197998047 nb_pixel_total : 24771 time to create 1 rle with old method : 0.03034687042236328 length of segment : 339 time for calcul the mask position with numpy : 0.012752771377563477 nb_pixel_total : 376481 time to create 1 rle with new method : 0.0336301326751709 length of segment : 691 time for calcul the mask position with numpy : 0.0011341571807861328 nb_pixel_total : 54564 time to create 1 rle with old method : 0.06220412254333496 length of segment : 321 time for calcul the mask position with numpy : 0.00027680397033691406 nb_pixel_total : 13436 time to create 1 rle with old method : 0.01602458953857422 length of segment : 138 time for calcul the mask position with numpy : 0.0004918575286865234 nb_pixel_total : 28247 time to create 1 rle with old method : 0.033826351165771484 length of segment : 161 time for calcul the mask position with numpy : 0.0036928653717041016 nb_pixel_total : 218024 time to create 1 rle with new method : 0.014185905456542969 length of segment : 639 time for calcul the mask position with numpy : 0.000919342041015625 nb_pixel_total : 35406 time to create 1 rle with old method : 0.042555809020996094 length of segment : 224 time for calcul the mask position with numpy : 0.00138092041015625 nb_pixel_total : 64328 time to create 1 rle with old method : 0.09038090705871582 length of segment : 467 time for calcul the mask position with numpy : 0.004897117614746094 nb_pixel_total : 233070 time to create 1 rle with new method : 0.015172481536865234 length of segment : 768 time for calcul the mask position with numpy : 0.0014297962188720703 nb_pixel_total : 95919 time to create 1 rle with old method : 0.10888886451721191 length of segment : 447 time for calcul the mask position with numpy : 0.0015094280242919922 nb_pixel_total : 78795 time to create 1 rle with old method : 0.09041595458984375 length of segment : 492 time for calcul the mask position with numpy : 0.0003104209899902344 nb_pixel_total : 16174 time to create 1 rle with old method : 0.019475460052490234 length of segment : 150 time for calcul the mask position with numpy : 0.0009043216705322266 nb_pixel_total : 39922 time to create 1 rle with old method : 0.04710650444030762 length of segment : 280 time for calcul the mask position with numpy : 0.0013058185577392578 nb_pixel_total : 59086 time to create 1 rle with old method : 0.06929707527160645 length of segment : 442 time for calcul the mask position with numpy : 0.0021898746490478516 nb_pixel_total : 78889 time to create 1 rle with old method : 0.0930185317993164 length of segment : 472 time for calcul the mask position with numpy : 0.001249074935913086 nb_pixel_total : 59908 time to create 1 rle with old method : 0.06931543350219727 length of segment : 248 time for calcul the mask position with numpy : 0.0022013187408447266 nb_pixel_total : 115458 time to create 1 rle with old method : 0.13424086570739746 length of segment : 408 time for calcul the mask position with numpy : 0.0008852481842041016 nb_pixel_total : 24828 time to create 1 rle with old method : 0.028980255126953125 length of segment : 242 time for calcul the mask position with numpy : 0.0023453235626220703 nb_pixel_total : 105057 time to create 1 rle with old method : 0.11978745460510254 length of segment : 346 time for calcul the mask position with numpy : 0.0008363723754882812 nb_pixel_total : 45169 time to create 1 rle with old method : 0.05890703201293945 length of segment : 241 time for calcul the mask position with numpy : 0.0045168399810791016 nb_pixel_total : 184975 time to create 1 rle with new method : 0.010438680648803711 length of segment : 767 time for calcul the mask position with numpy : 0.000659942626953125 nb_pixel_total : 32794 time to create 1 rle with old method : 0.03977680206298828 length of segment : 204 time for calcul the mask position with numpy : 0.0007507801055908203 nb_pixel_total : 52873 time to create 1 rle with old method : 0.0605015754699707 length of segment : 235 time for calcul the mask position with numpy : 0.004393577575683594 nb_pixel_total : 260785 time to create 1 rle with new method : 0.014051437377929688 length of segment : 758 time for calcul the mask position with numpy : 0.00025725364685058594 nb_pixel_total : 8701 time to create 1 rle with old method : 0.010479927062988281 length of segment : 127 time for calcul the mask position with numpy : 0.00072479248046875 nb_pixel_total : 36075 time to create 1 rle with old method : 0.04393815994262695 length of segment : 296 time for calcul the mask position with numpy : 0.0013308525085449219 nb_pixel_total : 53658 time to create 1 rle with old method : 0.06182384490966797 length of segment : 329 time for calcul the mask position with numpy : 0.0012993812561035156 nb_pixel_total : 62448 time to create 1 rle with old method : 0.07145118713378906 length of segment : 369 time for calcul the mask position with numpy : 0.00052642822265625 nb_pixel_total : 26087 time to create 1 rle with old method : 0.034310340881347656 length of segment : 230 time for calcul the mask position with numpy : 0.0006439685821533203 nb_pixel_total : 25843 time to create 1 rle with old method : 0.030071735382080078 length of segment : 208 time for calcul the mask position with numpy : 0.00041675567626953125 nb_pixel_total : 23084 time to create 1 rle with old method : 0.028158903121948242 length of segment : 131 time for calcul the mask position with numpy : 0.0004115104675292969 nb_pixel_total : 16343 time to create 1 rle with old method : 0.01930403709411621 length of segment : 188 time for calcul the mask position with numpy : 0.0019230842590332031 nb_pixel_total : 97525 time to create 1 rle with old method : 0.11315345764160156 length of segment : 314 time for calcul the mask position with numpy : 0.0009458065032958984 nb_pixel_total : 33629 time to create 1 rle with old method : 0.040229082107543945 length of segment : 245 time for calcul the mask position with numpy : 0.002518892288208008 nb_pixel_total : 90681 time to create 1 rle with old method : 0.1076819896697998 length of segment : 403 time for calcul the mask position with numpy : 0.0004684925079345703 nb_pixel_total : 18792 time to create 1 rle with old method : 0.021557092666625977 length of segment : 156 time for calcul the mask position with numpy : 0.0018801689147949219 nb_pixel_total : 61902 time to create 1 rle with old method : 0.07304263114929199 length of segment : 185 time for calcul the mask position with numpy : 0.001138925552368164 nb_pixel_total : 25053 time to create 1 rle with old method : 0.029413938522338867 length of segment : 175 time for calcul the mask position with numpy : 0.0012736320495605469 nb_pixel_total : 35792 time to create 1 rle with old method : 0.04150676727294922 length of segment : 235 time for calcul the mask position with numpy : 0.0014908313751220703 nb_pixel_total : 41603 time to create 1 rle with old method : 0.048493385314941406 length of segment : 274 time for calcul the mask position with numpy : 0.0008194446563720703 nb_pixel_total : 28358 time to create 1 rle with old method : 0.033629655838012695 length of segment : 168 time for calcul the mask position with numpy : 0.0026874542236328125 nb_pixel_total : 62801 time to create 1 rle with old method : 0.07528519630432129 length of segment : 584 time for calcul the mask position with numpy : 0.00139617919921875 nb_pixel_total : 49291 time to create 1 rle with old method : 0.056540489196777344 length of segment : 214 time for calcul the mask position with numpy : 0.003991127014160156 nb_pixel_total : 105352 time to create 1 rle with old method : 0.1197359561920166 length of segment : 493 time for calcul the mask position with numpy : 0.0017733573913574219 nb_pixel_total : 44422 time to create 1 rle with old method : 0.051245927810668945 length of segment : 262 time for calcul the mask position with numpy : 0.0059642791748046875 nb_pixel_total : 266373 time to create 1 rle with new method : 0.011301040649414062 length of segment : 624 time for calcul the mask position with numpy : 0.005059719085693359 nb_pixel_total : 251334 time to create 1 rle with new method : 0.009782791137695312 length of segment : 440 time for calcul the mask position with numpy : 0.003436565399169922 nb_pixel_total : 144991 time to create 1 rle with old method : 0.1645522117614746 length of segment : 286 time for calcul the mask position with numpy : 0.003889322280883789 nb_pixel_total : 125386 time to create 1 rle with old method : 0.157517671585083 length of segment : 356 time for calcul the mask position with numpy : 0.0007908344268798828 nb_pixel_total : 29463 time to create 1 rle with old method : 0.03452944755554199 length of segment : 163 time for calcul the mask position with numpy : 0.0013566017150878906 nb_pixel_total : 43100 time to create 1 rle with old method : 0.05055499076843262 length of segment : 307 time for calcul the mask position with numpy : 0.0007007122039794922 nb_pixel_total : 22587 time to create 1 rle with old method : 0.026472091674804688 length of segment : 161 time for calcul the mask position with numpy : 0.0003266334533691406 nb_pixel_total : 15125 time to create 1 rle with old method : 0.017426490783691406 length of segment : 112 time for calcul the mask position with numpy : 0.0031015872955322266 nb_pixel_total : 102744 time to create 1 rle with old method : 0.11398530006408691 length of segment : 378 time for calcul the mask position with numpy : 0.0021948814392089844 nb_pixel_total : 71851 time to create 1 rle with old method : 0.0816807746887207 length of segment : 476 time for calcul the mask position with numpy : 0.0012621879577636719 nb_pixel_total : 29631 time to create 1 rle with old method : 0.03333306312561035 length of segment : 322 time for calcul the mask position with numpy : 0.0012905597686767578 nb_pixel_total : 28433 time to create 1 rle with old method : 0.041983604431152344 length of segment : 267 time for calcul the mask position with numpy : 0.0010159015655517578 nb_pixel_total : 13190 time to create 1 rle with old method : 0.01991558074951172 length of segment : 195 time for calcul the mask position with numpy : 0.0010738372802734375 nb_pixel_total : 35816 time to create 1 rle with old method : 0.040770769119262695 length of segment : 261 time for calcul the mask position with numpy : 0.003975868225097656 nb_pixel_total : 121122 time to create 1 rle with old method : 0.13694000244140625 length of segment : 414 time for calcul the mask position with numpy : 0.00038743019104003906 nb_pixel_total : 14062 time to create 1 rle with old method : 0.016550779342651367 length of segment : 89 time for calcul the mask position with numpy : 0.00528264045715332 nb_pixel_total : 168816 time to create 1 rle with new method : 0.010054588317871094 length of segment : 305 time for calcul the mask position with numpy : 0.003448486328125 nb_pixel_total : 108361 time to create 1 rle with old method : 0.12601757049560547 length of segment : 448 time for calcul the mask position with numpy : 0.0023157596588134766 nb_pixel_total : 73690 time to create 1 rle with old method : 0.08313989639282227 length of segment : 484 time for calcul the mask position with numpy : 0.0009982585906982422 nb_pixel_total : 27566 time to create 1 rle with old method : 0.03141379356384277 length of segment : 213 time for calcul the mask position with numpy : 0.0008757114410400391 nb_pixel_total : 18578 time to create 1 rle with old method : 0.021510601043701172 length of segment : 264 time for calcul the mask position with numpy : 0.0013625621795654297 nb_pixel_total : 53842 time to create 1 rle with old method : 0.05984830856323242 length of segment : 220 time for calcul the mask position with numpy : 0.003153085708618164 nb_pixel_total : 121580 time to create 1 rle with old method : 0.13796210289001465 length of segment : 281 time for calcul the mask position with numpy : 0.0018579959869384766 nb_pixel_total : 58522 time to create 1 rle with old method : 0.0679624080657959 length of segment : 356 time for calcul the mask position with numpy : 0.0004935264587402344 nb_pixel_total : 25192 time to create 1 rle with old method : 0.029410362243652344 length of segment : 240 time for calcul the mask position with numpy : 0.0037679672241210938 nb_pixel_total : 103856 time to create 1 rle with old method : 0.11587953567504883 length of segment : 476 time for calcul the mask position with numpy : 0.0036301612854003906 nb_pixel_total : 167702 time to create 1 rle with new method : 0.006582975387573242 length of segment : 492 time for calcul the mask position with numpy : 0.0006237030029296875 nb_pixel_total : 24847 time to create 1 rle with old method : 0.028223752975463867 length of segment : 169 time for calcul the mask position with numpy : 0.0002987384796142578 nb_pixel_total : 11656 time to create 1 rle with old method : 0.013895750045776367 length of segment : 159 time for calcul the mask position with numpy : 0.0005102157592773438 nb_pixel_total : 14145 time to create 1 rle with old method : 0.01614665985107422 length of segment : 152 time for calcul the mask position with numpy : 0.001438140869140625 nb_pixel_total : 38515 time to create 1 rle with old method : 0.04388570785522461 length of segment : 282 time for calcul the mask position with numpy : 0.0004875659942626953 nb_pixel_total : 10819 time to create 1 rle with old method : 0.012744426727294922 length of segment : 139 time for calcul the mask position with numpy : 0.0014264583587646484 nb_pixel_total : 65405 time to create 1 rle with old method : 0.07743620872497559 length of segment : 220 time for calcul the mask position with numpy : 0.002888202667236328 nb_pixel_total : 96952 time to create 1 rle with old method : 0.1106722354888916 length of segment : 405 time for calcul the mask position with numpy : 0.004674434661865234 nb_pixel_total : 143777 time to create 1 rle with old method : 0.1655423641204834 length of segment : 435 time for calcul the mask position with numpy : 0.003531932830810547 nb_pixel_total : 138311 time to create 1 rle with old method : 0.15683650970458984 length of segment : 407 time for calcul the mask position with numpy : 0.0050089359283447266 nb_pixel_total : 161827 time to create 1 rle with new method : 0.007757663726806641 length of segment : 551 time for calcul the mask position with numpy : 0.0042572021484375 nb_pixel_total : 151465 time to create 1 rle with new method : 0.007373332977294922 length of segment : 507 time for calcul the mask position with numpy : 0.0009579658508300781 nb_pixel_total : 24430 time to create 1 rle with old method : 0.02853870391845703 length of segment : 214 time for calcul the mask position with numpy : 0.0012362003326416016 nb_pixel_total : 37735 time to create 1 rle with old method : 0.0432891845703125 length of segment : 235 time for calcul the mask position with numpy : 0.0053479671478271484 nb_pixel_total : 168232 time to create 1 rle with new method : 0.008526802062988281 length of segment : 603 time for calcul the mask position with numpy : 0.002482175827026367 nb_pixel_total : 94434 time to create 1 rle with old method : 0.10913562774658203 length of segment : 353 time for calcul the mask position with numpy : 0.0007102489471435547 nb_pixel_total : 16814 time to create 1 rle with old method : 0.01971435546875 length of segment : 132 time for calcul the mask position with numpy : 0.005328655242919922 nb_pixel_total : 167739 time to create 1 rle with new method : 0.011625528335571289 length of segment : 766 time for calcul the mask position with numpy : 0.0008041858673095703 nb_pixel_total : 25546 time to create 1 rle with old method : 0.029990434646606445 length of segment : 109 time for calcul the mask position with numpy : 0.0007073879241943359 nb_pixel_total : 19402 time to create 1 rle with old method : 0.02281022071838379 length of segment : 155 time for calcul the mask position with numpy : 0.0007638931274414062 nb_pixel_total : 28354 time to create 1 rle with old method : 0.03402853012084961 length of segment : 163 time for calcul the mask position with numpy : 0.0008134841918945312 nb_pixel_total : 12871 time to create 1 rle with old method : 0.015449285507202148 length of segment : 221 time for calcul the mask position with numpy : 0.0018172264099121094 nb_pixel_total : 73869 time to create 1 rle with old method : 0.08467864990234375 length of segment : 307 time for calcul the mask position with numpy : 0.0029349327087402344 nb_pixel_total : 79070 time to create 1 rle with old method : 0.09473180770874023 length of segment : 568 time for calcul the mask position with numpy : 0.0005087852478027344 nb_pixel_total : 13048 time to create 1 rle with old method : 0.01601696014404297 length of segment : 81 time for calcul the mask position with numpy : 0.004336833953857422 nb_pixel_total : 189434 time to create 1 rle with new method : 0.009897708892822266 length of segment : 545 time for calcul the mask position with numpy : 0.0014252662658691406 nb_pixel_total : 47409 time to create 1 rle with old method : 0.05290079116821289 length of segment : 349 time for calcul the mask position with numpy : 0.0009677410125732422 nb_pixel_total : 43300 time to create 1 rle with old method : 0.048020124435424805 length of segment : 242 time for calcul the mask position with numpy : 0.0008034706115722656 nb_pixel_total : 15585 time to create 1 rle with old method : 0.01804518699645996 length of segment : 220 time for calcul the mask position with numpy : 0.00079345703125 nb_pixel_total : 22009 time to create 1 rle with old method : 0.024827957153320312 length of segment : 168 time for calcul the mask position with numpy : 0.006144285202026367 nb_pixel_total : 118029 time to create 1 rle with old method : 0.14429998397827148 length of segment : 514 time for calcul the mask position with numpy : 0.0008406639099121094 nb_pixel_total : 21445 time to create 1 rle with old method : 0.025334596633911133 length of segment : 197 time for calcul the mask position with numpy : 0.0025415420532226562 nb_pixel_total : 83169 time to create 1 rle with old method : 0.09442543983459473 length of segment : 422 time for calcul the mask position with numpy : 0.0002751350402832031 nb_pixel_total : 6375 time to create 1 rle with old method : 0.007568359375 length of segment : 94 time for calcul the mask position with numpy : 0.004906892776489258 nb_pixel_total : 172826 time to create 1 rle with new method : 0.008636713027954102 length of segment : 507 time for calcul the mask position with numpy : 0.004610300064086914 nb_pixel_total : 167004 time to create 1 rle with new method : 0.0071604251861572266 length of segment : 534 time for calcul the mask position with numpy : 0.0007147789001464844 nb_pixel_total : 17373 time to create 1 rle with old method : 0.020112276077270508 length of segment : 217 time for calcul the mask position with numpy : 0.001531362533569336 nb_pixel_total : 42954 time to create 1 rle with old method : 0.049875497817993164 length of segment : 227 time for calcul the mask position with numpy : 0.0013515949249267578 nb_pixel_total : 31988 time to create 1 rle with old method : 0.03754591941833496 length of segment : 174 time for calcul the mask position with numpy : 0.0017931461334228516 nb_pixel_total : 61286 time to create 1 rle with old method : 0.07239532470703125 length of segment : 213 time for calcul the mask position with numpy : 0.0029768943786621094 nb_pixel_total : 176415 time to create 1 rle with new method : 0.009531021118164062 length of segment : 543 time for calcul the mask position with numpy : 0.0015797615051269531 nb_pixel_total : 39076 time to create 1 rle with old method : 0.04519915580749512 length of segment : 303 time for calcul the mask position with numpy : 0.001752614974975586 nb_pixel_total : 62480 time to create 1 rle with old method : 0.07232141494750977 length of segment : 473 time for calcul the mask position with numpy : 0.0009508132934570312 nb_pixel_total : 27970 time to create 1 rle with old method : 0.032562971115112305 length of segment : 218 time for calcul the mask position with numpy : 0.0035691261291503906 nb_pixel_total : 93901 time to create 1 rle with old method : 0.12141299247741699 length of segment : 350 time for calcul the mask position with numpy : 0.001352548599243164 nb_pixel_total : 31130 time to create 1 rle with old method : 0.03713655471801758 length of segment : 221 time spent for convertir_results : 13.517624139785767 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 120 chid ids of type : 3594 Number RLEs to save : 38952 save missing photos in datou_result : time spend for datou_step_exec : 83.14394068717957 time spend to save output : 2.4162979125976562 total time spend for step 1 : 85.56023859977722 step2:crop_condition Mon May 19 19:41:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 120 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 69 About to insert : list_path_to_insert length 69 new photo from crops ! About to upload 69 photos upload in portfolio : 3736932 init cache_photo without model_param we have 69 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747676539_3190892 we have uploaded 69 photos in the portfolio 3736932 time of upload the photos Elapsed time : 23.229726552963257 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3736932 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747676564_3190892 we have uploaded 9 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.5472495555877686 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747676569_3190892 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.9966552257537842 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 33 About to insert : list_path_to_insert length 33 new photo from crops ! About to upload 33 photos upload in portfolio : 3736932 init cache_photo without model_param we have 33 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747676588_3190892 we have uploaded 33 photos in the portfolio 3736932 time of upload the photos Elapsed time : 18.046767711639404 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747676608_3190892 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.0120916366577148 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747676610_3190892 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5978367328643799 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359488841, 1359488808, 1359488758, 1359488721, 1359488700, 1359488636, 1359487983, 1359487958] Looping around the photos to save general results len do output : 120 /1359499365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359499987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359500002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488841', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488808', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488758', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488721', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488700', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488636', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487983', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487958', None, None, None, None, None, '2898167') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 368 time used for this insertion : 0.02944183349609375 save_final save missing photos in datou_result : time spend for datou_step_exec : 92.60422921180725 time spend to save output : 0.034354209899902344 total time spend for step 2 : 92.63858342170715 step3:rle_unique_nms_with_priority Mon May 19 19:43:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 120 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 15 nb_hashtags : 4 time to prepare the origin masks : 8.348702192306519 time for calcul the mask position with numpy : 1.7755646705627441 nb_pixel_total : 6967213 time to create 1 rle with new method : 1.7465181350708008 time for calcul the mask position with numpy : 0.04510164260864258 nb_pixel_total : 16174 time to create 1 rle with old method : 0.01863384246826172 time for calcul the mask position with numpy : 0.04297685623168945 nb_pixel_total : 78795 time to create 1 rle with old method : 0.09100651741027832 time for calcul the mask position with numpy : 0.03995347023010254 nb_pixel_total : 14995 time to create 1 rle with old method : 0.017676830291748047 time for calcul the mask position with numpy : 0.03200793266296387 nb_pixel_total : 219064 time to create 1 rle with new method : 0.8369615077972412 time for calcul the mask position with numpy : 0.0282442569732666 nb_pixel_total : 64328 time to create 1 rle with old method : 0.07429695129394531 time for calcul the mask position with numpy : 0.028539657592773438 nb_pixel_total : 35292 time to create 1 rle with old method : 0.041600704193115234 time for calcul the mask position with numpy : 0.028293371200561523 nb_pixel_total : 218024 time to create 1 rle with new method : 2.5888826847076416 time for calcul the mask position with numpy : 0.0263519287109375 nb_pixel_total : 28247 time to create 1 rle with old method : 0.032047271728515625 time for calcul the mask position with numpy : 0.02664923667907715 nb_pixel_total : 13436 time to create 1 rle with old method : 0.015758275985717773 time for calcul the mask position with numpy : 0.025920391082763672 nb_pixel_total : 54564 time to create 1 rle with old method : 0.06235003471374512 time for calcul the mask position with numpy : 0.028169870376586914 nb_pixel_total : 376481 time to create 1 rle with new method : 0.9170687198638916 time for calcul the mask position with numpy : 0.025341272354125977 nb_pixel_total : 24771 time to create 1 rle with old method : 0.028995752334594727 time for calcul the mask position with numpy : 0.026118040084838867 nb_pixel_total : 82518 time to create 1 rle with old method : 0.09394145011901855 time for calcul the mask position with numpy : 0.033393144607543945 nb_pixel_total : 71720 time to create 1 rle with old method : 0.08351612091064453 time for calcul the mask position with numpy : 0.04069089889526367 nb_pixel_total : 28778 time to create 1 rle with old method : 0.032449960708618164 create new chi : 9.053850412368774 time to delete rle : 0.01991415023803711 batch 1 Loaded 31 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 13482 TO DO : save crop sub photo not yet done ! save time : 0.9691140651702881 nb_obj : 15 nb_hashtags : 4 time to prepare the origin masks : 11.228390455245972 time for calcul the mask position with numpy : 0.7548544406890869 nb_pixel_total : 7319685 time to create 1 rle with new method : 1.2468299865722656 time for calcul the mask position with numpy : 0.03528094291687012 nb_pixel_total : 53658 time to create 1 rle with old method : 0.06304526329040527 time for calcul the mask position with numpy : 0.02579188346862793 nb_pixel_total : 36075 time to create 1 rle with old method : 0.04205584526062012 time for calcul the mask position with numpy : 0.025363445281982422 nb_pixel_total : 7216 time to create 1 rle with old method : 0.008350610733032227 time for calcul the mask position with numpy : 0.025296926498413086 nb_pixel_total : 78807 time to create 1 rle with old method : 0.09060525894165039 time for calcul the mask position with numpy : 0.024326801300048828 nb_pixel_total : 52873 time to create 1 rle with old method : 0.06110048294067383 time for calcul the mask position with numpy : 0.025864839553833008 nb_pixel_total : 32794 time to create 1 rle with old method : 0.03796577453613281 time for calcul the mask position with numpy : 0.02542734146118164 nb_pixel_total : 184975 time to create 1 rle with new method : 0.9489352703094482 time for calcul the mask position with numpy : 0.026375532150268555 nb_pixel_total : 45169 time to create 1 rle with old method : 0.052153825759887695 time for calcul the mask position with numpy : 0.026696205139160156 nb_pixel_total : 105057 time to create 1 rle with old method : 0.12082982063293457 time for calcul the mask position with numpy : 0.025429487228393555 nb_pixel_total : 24828 time to create 1 rle with old method : 0.028664112091064453 time for calcul the mask position with numpy : 0.025502681732177734 nb_pixel_total : 115458 time to create 1 rle with old method : 0.1569674015045166 time for calcul the mask position with numpy : 0.027311325073242188 nb_pixel_total : 59908 time to create 1 rle with old method : 0.0702509880065918 time for calcul the mask position with numpy : 0.028354883193969727 nb_pixel_total : 78889 time to create 1 rle with old method : 0.09270834922790527 time for calcul the mask position with numpy : 0.02857518196105957 nb_pixel_total : 59086 time to create 1 rle with old method : 0.07261872291564941 time for calcul the mask position with numpy : 0.026170969009399414 nb_pixel_total : 39922 time to create 1 rle with old method : 0.0473477840423584 create new chi : 4.372126579284668 time to delete rle : 0.0018031597137451172 batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 12542 TO DO : save crop sub photo not yet done ! save time : 0.8183999061584473 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 11.675381898880005 time for calcul the mask position with numpy : 0.5312540531158447 nb_pixel_total : 7489816 time to create 1 rle with new method : 1.0071415901184082 time for calcul the mask position with numpy : 0.03471255302429199 nb_pixel_total : 105352 time to create 1 rle with old method : 0.1189875602722168 time for calcul the mask position with numpy : 0.03570055961608887 nb_pixel_total : 49291 time to create 1 rle with old method : 0.05696511268615723 time for calcul the mask position with numpy : 0.0297544002532959 nb_pixel_total : 62801 time to create 1 rle with old method : 0.07202363014221191 time for calcul the mask position with numpy : 0.04232478141784668 nb_pixel_total : 28358 time to create 1 rle with old method : 0.032944440841674805 time for calcul the mask position with numpy : 0.044019222259521484 nb_pixel_total : 41603 time to create 1 rle with old method : 0.06421303749084473 time for calcul the mask position with numpy : 0.04570651054382324 nb_pixel_total : 35792 time to create 1 rle with old method : 0.04035305976867676 time for calcul the mask position with numpy : 0.04030013084411621 nb_pixel_total : 25053 time to create 1 rle with old method : 0.028944969177246094 time for calcul the mask position with numpy : 0.04041242599487305 nb_pixel_total : 61902 time to create 1 rle with old method : 0.07159638404846191 time for calcul the mask position with numpy : 0.040914297103881836 nb_pixel_total : 18792 time to create 1 rle with old method : 0.022027015686035156 time for calcul the mask position with numpy : 0.04000234603881836 nb_pixel_total : 90681 time to create 1 rle with old method : 0.10534930229187012 time for calcul the mask position with numpy : 0.04418206214904785 nb_pixel_total : 33629 time to create 1 rle with old method : 0.03957653045654297 time for calcul the mask position with numpy : 0.04182147979736328 nb_pixel_total : 97525 time to create 1 rle with old method : 0.11453485488891602 time for calcul the mask position with numpy : 0.0408022403717041 nb_pixel_total : 16343 time to create 1 rle with old method : 0.019025087356567383 time for calcul the mask position with numpy : 0.03003859519958496 nb_pixel_total : 23084 time to create 1 rle with old method : 0.02716660499572754 time for calcul the mask position with numpy : 0.02455878257751465 nb_pixel_total : 25843 time to create 1 rle with old method : 0.02961421012878418 time for calcul the mask position with numpy : 0.02395319938659668 nb_pixel_total : 26087 time to create 1 rle with old method : 0.02951335906982422 time for calcul the mask position with numpy : 0.026298046112060547 nb_pixel_total : 62448 time to create 1 rle with old method : 0.0709688663482666 create new chi : 3.1593546867370605 time to delete rle : 0.001756906509399414 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 11304 TO DO : save crop sub photo not yet done ! save time : 0.7325785160064697 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 8.74098253250122 time for calcul the mask position with numpy : 0.5501670837402344 nb_pixel_total : 7155365 time to create 1 rle with new method : 0.8701775074005127 time for calcul the mask position with numpy : 0.0394434928894043 nb_pixel_total : 29631 time to create 1 rle with old method : 0.0342559814453125 time for calcul the mask position with numpy : 0.045298099517822266 nb_pixel_total : 71851 time to create 1 rle with old method : 0.10601186752319336 time for calcul the mask position with numpy : 0.04286956787109375 nb_pixel_total : 102744 time to create 1 rle with old method : 0.11688089370727539 time for calcul the mask position with numpy : 0.04051828384399414 nb_pixel_total : 7153 time to create 1 rle with old method : 0.008365154266357422 time for calcul the mask position with numpy : 0.04078316688537598 nb_pixel_total : 22587 time to create 1 rle with old method : 0.02587270736694336 time for calcul the mask position with numpy : 0.03894686698913574 nb_pixel_total : 43100 time to create 1 rle with old method : 0.04955029487609863 time for calcul the mask position with numpy : 0.03881192207336426 nb_pixel_total : 29463 time to create 1 rle with old method : 0.03399538993835449 time for calcul the mask position with numpy : 0.03928804397583008 nb_pixel_total : 125386 time to create 1 rle with old method : 0.14290595054626465 time for calcul the mask position with numpy : 0.03916049003601074 nb_pixel_total : 144991 time to create 1 rle with old method : 0.1658153533935547 time for calcul the mask position with numpy : 0.03655719757080078 nb_pixel_total : 251334 time to create 1 rle with new method : 0.8905029296875 time for calcul the mask position with numpy : 0.028680801391601562 nb_pixel_total : 266373 time to create 1 rle with new method : 0.8570601940155029 time for calcul the mask position with numpy : 0.04263949394226074 nb_pixel_total : 44422 time to create 1 rle with old method : 0.05150008201599121 create new chi : 4.484444856643677 time to delete rle : 0.0024149417877197266 batch 1 Loaded 25 chid ids of type : 3594 +++++++++++++Number RLEs to save : 9835 TO DO : save crop sub photo not yet done ! save time : 0.6729350090026855 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 6.694916725158691 time for calcul the mask position with numpy : 0.7983710765838623 nb_pixel_total : 7630980 time to create 1 rle with new method : 1.001936674118042 time for calcul the mask position with numpy : 0.023067474365234375 nb_pixel_total : 53842 time to create 1 rle with old method : 0.059439659118652344 time for calcul the mask position with numpy : 0.023066282272338867 nb_pixel_total : 18578 time to create 1 rle with old method : 0.020228147506713867 time for calcul the mask position with numpy : 0.024020671844482422 nb_pixel_total : 27566 time to create 1 rle with old method : 0.03100275993347168 time for calcul the mask position with numpy : 0.02457427978515625 nb_pixel_total : 73634 time to create 1 rle with old method : 0.08160209655761719 time for calcul the mask position with numpy : 0.023947715759277344 nb_pixel_total : 108361 time to create 1 rle with old method : 0.12236928939819336 time for calcul the mask position with numpy : 0.02489757537841797 nb_pixel_total : 168816 time to create 1 rle with new method : 0.858342170715332 time for calcul the mask position with numpy : 0.026649951934814453 nb_pixel_total : 14062 time to create 1 rle with old method : 0.01622295379638672 time for calcul the mask position with numpy : 0.0263211727142334 nb_pixel_total : 121122 time to create 1 rle with old method : 0.13719415664672852 time for calcul the mask position with numpy : 0.02390146255493164 nb_pixel_total : 35816 time to create 1 rle with old method : 0.04092979431152344 time for calcul the mask position with numpy : 0.023918628692626953 nb_pixel_total : 13190 time to create 1 rle with old method : 0.015336275100708008 time for calcul the mask position with numpy : 0.024466991424560547 nb_pixel_total : 28433 time to create 1 rle with old method : 0.032085418701171875 create new chi : 3.554014205932617 time to delete rle : 0.001321554183959961 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 8444 TO DO : save crop sub photo not yet done ! save time : 0.5416529178619385 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 5.5742199420928955 time for calcul the mask position with numpy : 0.8353502750396729 nb_pixel_total : 7652161 time to create 1 rle with new method : 0.7728652954101562 time for calcul the mask position with numpy : 0.023868799209594727 nb_pixel_total : 65405 time to create 1 rle with old method : 0.07328939437866211 time for calcul the mask position with numpy : 0.024060964584350586 nb_pixel_total : 10819 time to create 1 rle with old method : 0.012259721755981445 time for calcul the mask position with numpy : 0.024291276931762695 nb_pixel_total : 38515 time to create 1 rle with old method : 0.04349112510681152 time for calcul the mask position with numpy : 0.0236361026763916 nb_pixel_total : 14145 time to create 1 rle with old method : 0.015784740447998047 time for calcul the mask position with numpy : 0.024042367935180664 nb_pixel_total : 11656 time to create 1 rle with old method : 0.013235807418823242 time for calcul the mask position with numpy : 0.024392127990722656 nb_pixel_total : 24847 time to create 1 rle with old method : 0.0279996395111084 time for calcul the mask position with numpy : 0.024559974670410156 nb_pixel_total : 167702 time to create 1 rle with new method : 0.776787281036377 time for calcul the mask position with numpy : 0.025269508361816406 nb_pixel_total : 103856 time to create 1 rle with old method : 0.11873126029968262 time for calcul the mask position with numpy : 0.024702072143554688 nb_pixel_total : 25192 time to create 1 rle with old method : 0.02832508087158203 time for calcul the mask position with numpy : 0.025246858596801758 nb_pixel_total : 58522 time to create 1 rle with old method : 0.06590723991394043 time for calcul the mask position with numpy : 0.025003671646118164 nb_pixel_total : 121580 time to create 1 rle with old method : 0.13787078857421875 create new chi : 3.2603955268859863 time to delete rle : 0.0014271736145019531 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++Number RLEs to save : 8092 TO DO : save crop sub photo not yet done ! save time : 0.5371353626251221 nb_obj : 21 nb_hashtags : 4 time to prepare the origin masks : 11.709353685379028 time for calcul the mask position with numpy : 0.44390416145324707 nb_pixel_total : 6560661 time to create 1 rle with new method : 0.743340015411377 time for calcul the mask position with numpy : 0.0417330265045166 nb_pixel_total : 43300 time to create 1 rle with old method : 0.04947400093078613 time for calcul the mask position with numpy : 0.03948163986206055 nb_pixel_total : 47409 time to create 1 rle with old method : 0.055864572525024414 time for calcul the mask position with numpy : 0.04384779930114746 nb_pixel_total : 189434 time to create 1 rle with new method : 0.7612357139587402 time for calcul the mask position with numpy : 0.024881601333618164 nb_pixel_total : 12768 time to create 1 rle with old method : 0.014898300170898438 time for calcul the mask position with numpy : 0.027597427368164062 nb_pixel_total : 79070 time to create 1 rle with old method : 0.11786913871765137 time for calcul the mask position with numpy : 0.02931070327758789 nb_pixel_total : 73869 time to create 1 rle with old method : 0.09127998352050781 time for calcul the mask position with numpy : 0.025607585906982422 nb_pixel_total : 12871 time to create 1 rle with old method : 0.015151262283325195 time for calcul the mask position with numpy : 0.03001999855041504 nb_pixel_total : 28354 time to create 1 rle with old method : 0.03296637535095215 time for calcul the mask position with numpy : 0.025252103805541992 nb_pixel_total : 19402 time to create 1 rle with old method : 0.022274017333984375 time for calcul the mask position with numpy : 0.025394916534423828 nb_pixel_total : 25546 time to create 1 rle with old method : 0.029638051986694336 time for calcul the mask position with numpy : 0.026055097579956055 nb_pixel_total : 167739 time to create 1 rle with new method : 0.6598107814788818 time for calcul the mask position with numpy : 0.024338483810424805 nb_pixel_total : 16814 time to create 1 rle with old method : 0.018799304962158203 time for calcul the mask position with numpy : 0.026551008224487305 nb_pixel_total : 94434 time to create 1 rle with old method : 0.1086111068725586 time for calcul the mask position with numpy : 0.03154444694519043 nb_pixel_total : 168232 time to create 1 rle with new method : 0.7169966697692871 time for calcul the mask position with numpy : 0.024631738662719727 nb_pixel_total : 37735 time to create 1 rle with old method : 0.04193282127380371 time for calcul the mask position with numpy : 0.02488255500793457 nb_pixel_total : 24430 time to create 1 rle with old method : 0.027552127838134766 time for calcul the mask position with numpy : 0.02508544921875 nb_pixel_total : 151465 time to create 1 rle with new method : 0.7415196895599365 time for calcul the mask position with numpy : 0.027444839477539062 nb_pixel_total : 161827 time to create 1 rle with new method : 0.9332790374755859 time for calcul the mask position with numpy : 0.026761293411254883 nb_pixel_total : 138311 time to create 1 rle with old method : 0.18331408500671387 time for calcul the mask position with numpy : 0.0265350341796875 nb_pixel_total : 143777 time to create 1 rle with old method : 0.16360235214233398 time for calcul the mask position with numpy : 0.026209115982055664 nb_pixel_total : 96952 time to create 1 rle with old method : 0.11660480499267578 create new chi : 6.869711399078369 time to delete rle : 0.0029594898223876953 batch 1 Loaded 43 chid ids of type : 3594 +++++++++++++++++++++++++++++++++Number RLEs to save : 16818 TO DO : save crop sub photo not yet done ! save time : 1.055488109588623 nb_obj : 18 nb_hashtags : 4 time to prepare the origin masks : 9.176860809326172 time for calcul the mask position with numpy : 0.6339473724365234 nb_pixel_total : 7267718 time to create 1 rle with new method : 2.0966291427612305 time for calcul the mask position with numpy : 0.04032015800476074 nb_pixel_total : 31130 time to create 1 rle with old method : 0.03574013710021973 time for calcul the mask position with numpy : 0.04031634330749512 nb_pixel_total : 93901 time to create 1 rle with old method : 0.10777664184570312 time for calcul the mask position with numpy : 0.03860807418823242 nb_pixel_total : 27970 time to create 1 rle with old method : 0.030530929565429688 time for calcul the mask position with numpy : 0.03754830360412598 nb_pixel_total : 62480 time to create 1 rle with old method : 0.07016849517822266 time for calcul the mask position with numpy : 0.04194760322570801 nb_pixel_total : 39076 time to create 1 rle with old method : 0.04487872123718262 time for calcul the mask position with numpy : 0.045740365982055664 nb_pixel_total : 12085 time to create 1 rle with old method : 0.01541280746459961 time for calcul the mask position with numpy : 0.04606366157531738 nb_pixel_total : 61286 time to create 1 rle with old method : 0.07393813133239746 time for calcul the mask position with numpy : 0.05060744285583496 nb_pixel_total : 31988 time to create 1 rle with old method : 0.05237579345703125 time for calcul the mask position with numpy : 0.05775928497314453 nb_pixel_total : 42954 time to create 1 rle with old method : 0.05314207077026367 time for calcul the mask position with numpy : 0.04320573806762695 nb_pixel_total : 17373 time to create 1 rle with old method : 0.020041942596435547 time for calcul the mask position with numpy : 0.04491567611694336 nb_pixel_total : 167004 time to create 1 rle with new method : 1.0868706703186035 time for calcul the mask position with numpy : 0.05169391632080078 nb_pixel_total : 172826 time to create 1 rle with new method : 0.6853063106536865 time for calcul the mask position with numpy : 0.03940391540527344 nb_pixel_total : 6375 time to create 1 rle with old method : 0.0073087215423583984 time for calcul the mask position with numpy : 0.03724050521850586 nb_pixel_total : 83166 time to create 1 rle with old method : 0.0954430103302002 time for calcul the mask position with numpy : 0.04477524757385254 nb_pixel_total : 21445 time to create 1 rle with old method : 0.028461933135986328 time for calcul the mask position with numpy : 0.04077768325805664 nb_pixel_total : 118029 time to create 1 rle with old method : 0.13151144981384277 time for calcul the mask position with numpy : 0.04038858413696289 nb_pixel_total : 22009 time to create 1 rle with old method : 0.024658679962158203 time for calcul the mask position with numpy : 0.04004955291748047 nb_pixel_total : 15585 time to create 1 rle with old method : 0.017446517944335938 create new chi : 6.1955084800720215 time to delete rle : 0.0031592845916748047 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++Number RLEs to save : 12958 TO DO : save crop sub photo not yet done ! save time : 0.8749628067016602 map_output_result : {1359488841: (0.0, 'Should be the crop_list due to order', 0), 1359488808: (0.0, 'Should be the crop_list due to order', 0), 1359488758: (0.0, 'Should be the crop_list due to order', 0), 1359488721: (0.0, 'Should be the crop_list due to order', 0), 1359488700: (0.0, 'Should be the crop_list due to order', 0), 1359488636: (0.0, 'Should be the crop_list due to order', 0), 1359487983: (0.0, 'Should be the crop_list due to order', 0), 1359487958: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359488841, 1359488808, 1359488758, 1359488721, 1359488700, 1359488636, 1359487983, 1359487958] Looping around the photos to save general results len do output : 8 /1359488841.Didn't retrieve data . /1359488808.Didn't retrieve data . /1359488758.Didn't retrieve data . /1359488721.Didn't retrieve data . /1359488700.Didn't retrieve data . /1359488636.Didn't retrieve data . /1359487983.Didn't retrieve data . /1359487958.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488841', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488808', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488758', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488721', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488700', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488636', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487983', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487958', None, None, None, None, None, '2898167') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.013982534408569336 save_final save missing photos in datou_result : time spend for datou_step_exec : 121.47425580024719 time spend to save output : 0.014436960220336914 total time spend for step 3 : 121.48869276046753 step4:ventilate_hashtags_in_portfolio Mon May 19 19:45:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23133812 get user id for portfolio 23133812 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23133812 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','mal_croppe','pet_fonce','flou','papier','background','carton','metal','pet_clair','autre','environnement')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23133812 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','mal_croppe','pet_fonce','flou','papier','background','carton','metal','pet_clair','autre','environnement')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23133812 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','mal_croppe','pet_fonce','flou','papier','background','carton','metal','pet_clair','autre','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23134564,23134565,23134566,23134567,23134568,23134569,23134570,23134571,23134572,23134573,23134574?tags=pehd,mal_croppe,pet_fonce,flou,papier,background,carton,metal,pet_clair,autre,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359488841, 1359488808, 1359488758, 1359488721, 1359488700, 1359488636, 1359487983, 1359487958] Looping around the photos to save general results len do output : 1 /23133812. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488841', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488808', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488758', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488721', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488700', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488636', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487983', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487958', None, None, None, None, None, '2898167') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.019330739974975586 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7877094745635986 time spend to save output : 0.01952528953552246 total time spend for step 4 : 0.8072347640991211 step5:final Mon May 19 19:45:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359488841: ('0.12525922610435958',), 1359488808: ('0.12525922610435958',), 1359488758: ('0.12525922610435958',), 1359488721: ('0.12525922610435958',), 1359488700: ('0.12525922610435958',), 1359488636: ('0.12525922610435958',), 1359487983: ('0.12525922610435958',), 1359487958: ('0.12525922610435958',)} new output for save of step final : {1359488841: ('0.12525922610435958',), 1359488808: ('0.12525922610435958',), 1359488758: ('0.12525922610435958',), 1359488721: ('0.12525922610435958',), 1359488700: ('0.12525922610435958',), 1359488636: ('0.12525922610435958',), 1359487983: ('0.12525922610435958',), 1359487958: ('0.12525922610435958',)} [1359488841, 1359488808, 1359488758, 1359488721, 1359488700, 1359488636, 1359487983, 1359487958] Looping around the photos to save general results len do output : 8 /1359488841.Didn't retrieve data . /1359488808.Didn't retrieve data . /1359488758.Didn't retrieve data . /1359488721.Didn't retrieve data . /1359488700.Didn't retrieve data . /1359488636.Didn't retrieve data . /1359487983.Didn't retrieve data . /1359487958.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488841', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488808', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488758', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488721', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488700', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488636', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487983', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487958', None, None, None, None, None, '2898167') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.013134956359863281 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10792684555053711 time spend to save output : 0.013573408126831055 total time spend for step 5 : 0.12150025367736816 step6:blur_detection Mon May 19 19:45:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b.jpg resize: (2160, 3840) 1359488841 -5.6778898504144735 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612.jpg resize: (2160, 3840) 1359488808 -5.651894223352588 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89.jpg resize: (2160, 3840) 1359488758 -5.6944320981435395 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0.jpg resize: (2160, 3840) 1359488721 -5.320017731588939 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae.jpg resize: (2160, 3840) 1359488700 -4.0535148147679605 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a.jpg resize: (2160, 3840) 1359488636 -5.6173555421099035 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643.jpg resize: (2160, 3840) 1359487983 -5.624828342601217 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1.jpg resize: (2160, 3840) 1359487958 -5.577132372370009 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525818_0.png resize: (161, 198) 1359499365 -2.2273208549887706 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525812_0.png resize: (376, 319) 1359499367 -3.6643802153610663 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525815_0.png resize: (658, 824) 1359499368 -2.4890877270088168 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525817_0.png resize: (136, 145) 1359499370 -0.3087927710450265 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525824_0.png resize: (391, 343) 1359499371 -3.836772763142517 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525816_0.png resize: (314, 300) 1359499372 -4.697040887091142 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525834_0.png resize: (622, 371) 1359499374 -2.5053009569249847 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525833_0.png resize: (181, 412) 1359499375 -4.025958842607408 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525838_0.png resize: (126, 118) 1359499377 -3.438448409889377 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525832_0.png resize: (346, 463) 1359499378 -1.3183085209761183 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525826_0.png resize: (280, 259) 1359499379 -2.2826607204409215 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525840_0.png resize: (347, 405) 1359499381 -3.849805028069411 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525829_0.png resize: (248, 381) 1359499382 -0.38005262184687627 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525828_0.png resize: (459, 337) 1359499385 -3.3945881853135464 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525839_0.png resize: (296, 177) 1359499388 -3.388249842102427 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525827_0.png resize: (442, 219) 1359499390 -2.4793775234329387 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525847_0.png resize: (245, 215) 1359499392 -2.727670023002795 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525855_0.png resize: (361, 342) 1359499393 -2.8579259725002553 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525850_0.png resize: (181, 521) 1359499397 -2.755093664911783 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525844_0.png resize: (129, 207) 1359499398 -0.23324811775782506 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525843_0.png resize: (208, 198) 1359499401 -1.4527876098008214 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525841_0.png resize: (330, 319) 1359499402 -3.3112729442203617 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525845_0.png resize: (187, 117) 1359499404 -3.3053296768705045 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525851_0.png resize: (159, 244) 1359499406 -3.488636914588179 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525868_0.png resize: (461, 214) 1359499408 -2.6751170555005506 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525860_0.png resize: (440, 637) 1359499410 -1.4787102200799382 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525869_0.png resize: (322, 174) 1359499411 -3.735283586993616 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525865_0.png resize: (161, 187) 1359499414 -4.113003799802115 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525858_0.png resize: (241, 349) 1359499415 -3.430647082714014 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525876_0.png resize: (381, 401) 1359499418 -4.3173473381880045 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525870_0.png resize: (267, 263) 1359499419 -2.598768082920073 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525879_0.png resize: (175, 207) 1359499420 -4.205911652534875 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525878_0.png resize: (220, 318) 1359499423 -3.112045533194941 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525871_0.png resize: (185, 136) 1359499424 -3.642704082691847 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525872_0.png resize: (240, 219) 1359499427 -3.330717300320682 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525877_0.png resize: (410, 303) 1359499428 -3.711639156321123 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525874_0.png resize: (87, 200) 1359499431 -3.3230807913952627 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525889_0.png resize: (266, 258) 1359499432 -4.50256307235655 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525887_0.png resize: (159, 110) 1359499433 -2.8949875298313534 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525888_0.png resize: (143, 124) 1359499436 -4.316940588285778 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525886_0.png resize: (168, 190) 1359499437 -2.674597312966456 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525883_0.png resize: (240, 143) 1359499440 -3.3013143731461616 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525890_0.png resize: (139, 129) 1359499441 -3.462579773464086 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525898_0.png resize: (235, 245) 1359499444 -1.1867154811270388 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525909_0.png resize: (64, 339) 1359499445 -3.8598086139613006 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525901_0.png resize: (132, 199) 1359499448 -4.032624351408174 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525896_0.png resize: (497, 398) 1359499449 -2.722247457068582 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525905_0.png resize: (161, 206) 1359499451 -4.068033082499418 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525911_0.png resize: (344, 212) 1359499453 -2.935278208161921 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525904_0.png resize: (155, 213) 1359499454 -3.4549190028041914 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525908_0.png resize: (423, 505) 1359499457 -1.7283247526415209 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525892_0.png resize: (404, 297) 1359499458 -3.9785544739501364 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525912_0.png resize: (241, 228) 1359499460 -3.212254045269065 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525894_0.png resize: (407, 489) 1359499462 -3.956462858323232 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525893_0.png resize: (435, 523) 1359499464 -4.9114674177570885 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525899_0.png resize: (550, 511) 1359499466 -3.500753576332181 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525907_0.png resize: (241, 437) 1359499467 -3.654844914693775 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525897_0.png resize: (214, 168) 1359499470 -4.27560853105254 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525922_0.png resize: (227, 305) 1359499471 -1.804802062378319 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525929_0.png resize: (336, 496) 1359499472 -3.656139552703706 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525927_0.png resize: (428, 234) 1359499475 -4.539744573293436 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525926_0.png resize: (280, 269) 1359499476 -2.87477204942571 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525921_0.png resize: (213, 115) 1359499479 -3.715790308770156 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525920_0.png resize: (511, 459) 1359499480 -4.209550837316816 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525915_0.png resize: (513, 497) 1359499483 -3.2866007828146264 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525913_0.png resize: (194, 118) 1359499484 -3.188031473573874 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525918_0.png resize: (94, 100) 1359499489 -3.0731360973139235 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525916_0.png resize: (189, 140) 1359499491 -3.727312986181665 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525914_0.png resize: (168, 168) 1359499492 -2.7021033930231373 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525811_0.png resize: (181, 228) 1359499543 -1.9806403272087028 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525831_0.png resize: (240, 186) 1359499546 -3.2568148429450936 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525842_0.png resize: (230, 147) 1359499548 -4.025867392180062 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525852_0.png resize: (232, 218) 1359499549 -2.5547945004463695 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525866_0.png resize: (112, 186) 1359499551 -3.562287339561877 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525863_0.png resize: (105, 330) 1359499552 -3.1699146236261653 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525880_0.png resize: (220, 346) 1359499554 -4.1144666796418425 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525882_0.png resize: (312, 267) 1359499556 -2.7388120077033036 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525906_0.png resize: (169, 164) 1359499557 -3.82682463739994 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525825_0.png resize: (148, 140) 1359499595 -4.341764309587372 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525823_0.png resize: (376, 309) 1359499598 -4.150013158892335 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525836_0.png resize: (211, 304) 1359499599 -3.469344458190051 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525864_0.png resize: (307, 189) 1359499600 -4.12583312432892 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525903_0.png resize: (108, 293) 1359499602 -4.739617016749628 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525928_0.png resize: (218, 158) 1359499603 -3.9825834687996764 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525821_0.png resize: (445, 267) 1359499890 -3.5781975930030563 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525822_0.png resize: (726, 429) 1359499892 -3.507522140181516 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525820_0.png resize: (223, 247) 1359499895 -2.7888438229016272 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525814_0.png resize: (241, 283) 1359499897 -3.1958008639907307 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525819_0.png resize: (570, 633) 1359499899 -4.034593381195828 treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525813_0.png resize: (355, 402) 1359499902 -3.978132685694513 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525837_0.png resize: (749, 516) 1359499904 -2.8889724505333065 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525830_0.png resize: (408, 441) 1359499907 -4.530596753640126 treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525835_0.png resize: (204, 263) 1359499910 -3.4618943814889684 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525846_0.png resize: (313, 423) 1359499911 -3.0594107143227207 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525853_0.png resize: (215, 297) 1359499914 -4.002191765238749 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525848_0.png resize: (375, 459) 1359499916 -4.1026161918128485 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525849_0.png resize: (156, 152) 1359499917 -3.0592110626232527 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525857_0.png resize: (493, 332) 1359499920 -3.2610705796262973 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525859_0.png resize: (624, 518) 1359499921 -4.3778077261338355 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525861_0.png resize: (286, 672) 1359499922 -3.592352208378526 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525862_0.png resize: (299, 600) 1359499925 -4.005913073357406 treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525867_0.png resize: (378, 435) 1359499927 -4.901175337965791 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525875_0.png resize: (283, 854) 1359499928 -3.1047287388163696 treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525873_0.png resize: (382, 585) 1359499931 -3.7728847202220113 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525881_0.png resize: (279, 591) 1359499932 -3.442745572388294 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525885_0.png resize: (489, 435) 1359499933 -5.051691069182036 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525891_0.png resize: (218, 401) 1359499936 -5.317169480540764 treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525884_0.png resize: (443, 447) 1359499937 -4.635264020645948 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525902_0.png resize: (766, 383) 1359499938 -3.886868835951053 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525900_0.png resize: (353, 349) 1359499941 -4.939489827893206 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525910_0.png resize: (540, 478) 1359499942 -3.977333385127492 treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525895_0.png resize: (551, 462) 1359499943 -3.7240847255336766 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525930_0.png resize: (216, 225) 1359499946 -3.2938824514782237 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525924_0.png resize: (203, 355) 1359499947 -3.8512502951631884 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525919_0.png resize: (462, 506) 1359499948 -4.095806462644038 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525925_0.png resize: (543, 435) 1359499950 -4.4384187798910295 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525917_0.png resize: (286, 464) 1359499952 -3.0305479381549643 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525856_0.png resize: (212, 299) 1359499986 -4.090364791145205 treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525854_0.png resize: (168, 200) 1359499987 -1.2999422087542047 treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525923_0.png resize: (162, 299) 1359500002 -3.1333150024682763 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 128 time used for this insertion : 0.022797107696533203 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 128 time used for this insertion : 0.030646085739135742 save missing photos in datou_result : time spend for datou_step_exec : 40.829713106155396 time spend to save output : 0.05924701690673828 total time spend for step 6 : 40.888960123062134 step7:brightness Mon May 19 19:46:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b.jpg treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612.jpg treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89.jpg treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0.jpg treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae.jpg treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a.jpg treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643.jpg treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1.jpg treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525818_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525812_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525815_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525817_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525824_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525816_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525834_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525833_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525838_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525832_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525826_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525840_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525829_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525828_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525839_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525827_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525847_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525855_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525850_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525844_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525843_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525841_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525845_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525851_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525868_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525860_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525869_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525865_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525858_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525876_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525870_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525879_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525878_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525871_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525872_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525877_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525874_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525889_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525887_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525888_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525886_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525883_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525890_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525898_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525909_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525901_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525896_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525905_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525911_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525904_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525908_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525892_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525912_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525894_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525893_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525899_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525907_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525897_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525922_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525929_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525927_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525926_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525921_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525920_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525915_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525913_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525918_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525916_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525914_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525811_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525831_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525842_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525852_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525866_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525863_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525880_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525882_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525906_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525825_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525823_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525836_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525864_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525903_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525928_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525821_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525822_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525820_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525814_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525819_0.png treat image : temp/1747676430_3190892_1359488841_1f03c60e51f011a6e196631a6ae9029b_rle_crop_3804525813_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525837_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525830_0.png treat image : temp/1747676430_3190892_1359488808_158d20d3fcc9abb3ba9b38ffe9685612_rle_crop_3804525835_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525846_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525853_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525848_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525849_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525857_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525859_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525861_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525862_0.png treat image : temp/1747676430_3190892_1359488721_b83bdf141a151df90da529782d3ab0c0_rle_crop_3804525867_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525875_0.png treat image : temp/1747676430_3190892_1359488700_d5985eada9d1b36cb53daac323b17cae_rle_crop_3804525873_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525881_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525885_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525891_0.png treat image : temp/1747676430_3190892_1359488636_29f0fd6bc51e2e17773895f0e5ed9e8a_rle_crop_3804525884_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525902_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525900_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525910_0.png treat image : temp/1747676430_3190892_1359487983_7cccbae41a10fa569b9591f71e2d8643_rle_crop_3804525895_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525930_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525924_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525919_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525925_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525917_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525856_0.png treat image : temp/1747676430_3190892_1359488758_37d67fa624a899e17db6169752b25c89_rle_crop_3804525854_0.png treat image : temp/1747676430_3190892_1359487958_71d3208d616a6d7b2d6c02c1c36b69a1_rle_crop_3804525923_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 128 time used for this insertion : 0.020458698272705078 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 128 time used for this insertion : 0.03179168701171875 save missing photos in datou_result : time spend for datou_step_exec : 9.319806814193726 time spend to save output : 0.05813407897949219 total time spend for step 7 : 9.377940893173218 step8:velours_tree Mon May 19 19:46:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.12934231758117676 time spend to save output : 4.839897155761719e-05 total time spend for step 8 : 0.12939071655273438 step9:send_mail_cod Mon May 19 19:46:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23133812_19-05-2025_19_46_23.pdf 23134564 imagette231345641747676783 23134565 imagette231345651747676783 23134566 change filename to text .imagette231345661747676783 23134567 imagette231345671747676783 23134568 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231345681747676783 23134569 imagette231345691747676784 23134570 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231345701747676784 23134571 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231345711747676785 23134572 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231345721747676786 23134573 change filename to text .change filename to text .imagette231345731747676788 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23133812 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23134564,23134565,23134566,23134567,23134568,23134569,23134570,23134571,23134572,23134573,23134574?tags=pehd,mal_croppe,pet_fonce,flou,papier,background,carton,metal,pet_clair,autre,environnement args[1359488841] : ((1359488841, -5.6778898504144735, 492609224), (1359488841, 0.3222233527220889, 2107752395), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359488808] : ((1359488808, -5.651894223352588, 492609224), (1359488808, 0.35891489446027397, 2107752395), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359488758] : ((1359488758, -5.6944320981435395, 492609224), (1359488758, 0.47047305295818065, 2107752395), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359488721] : ((1359488721, -5.320017731588939, 492609224), (1359488721, -0.07729758108120487, 496442774), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359488700] : ((1359488700, -4.0535148147679605, 492609224), (1359488700, -0.23650780113159112, 496442774), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359488636] : ((1359488636, -5.6173555421099035, 492609224), (1359488636, -0.1034327929146817, 496442774), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359487983] : ((1359487983, -5.624828342601217, 492609224), (1359487983, 0.22298592514467383, 2107752395), '0.12525922610435958') We are sending mail with results at report@fotonower.com args[1359487958] : ((1359487958, -5.577132372370009, 492609224), (1359487958, 0.3324957325125324, 2107752395), '0.12525922610435958') We are sending mail with results at report@fotonower.com refus_total : 0.12525922610435958 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23133812 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23133812_19-05-2025_19_46_23.pdf results_Auto_P23133812_19-05-2025_19_46_23.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23133812_19-05-2025_19_46_23.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23133812','results_Auto_P23133812_19-05-2025_19_46_23.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23133812_19-05-2025_19_46_23.pdf','pdf','','0.79','0.12525922610435958') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23133812

https://www.fotonower.com/image?json=false&list_photos_id=1359488841
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359488808
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359488758
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359488721
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359488700
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359488636
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359487983
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359487958
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 12.53%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23134566?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23134568?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23134570?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23134571?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23134572?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23134573?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23133812_19-05-2025_19_46_23.pdf.

Lien vers velours :https://www.fotonower.com/velours/23134564,23134565,23134566,23134567,23134568,23134569,23134570,23134571,23134572,23134573,23134574?tags=pehd,mal_croppe,pet_fonce,flou,papier,background,carton,metal,pet_clair,autre,environnement.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 19 May 2025 17:46:31 GMT Content-Length: 0 Connection: close X-Message-Id: 23rw0AqURYidn8Id9fuHEA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359488841, 1359488808, 1359488758, 1359488721, 1359488700, 1359488636, 1359487983, 1359487958] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488841', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488808', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488758', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488721', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488700', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488636', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487983', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487958', None, None, None, None, None, '2898167') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.013123035430908203 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.965079069137573 time spend to save output : 0.013344287872314453 total time spend for step 9 : 7.978423357009888 step10:split_time_score Mon May 19 19:46:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('19', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 19052025 23133812 Nombre de photos uploadées : 8 / 23040 (0%) 19052025 23133812 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 19052025 23133812 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.9604644775390625e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0004532337188720703 elapsed_time : insert_dashboard_record_day_entry 0.02360081672668457 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.12806391095562103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23104924_19-05-2025_10_08_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23104924 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23104924 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23104935 order by id desc limit 1 Qualite : 0.11080586682800225 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23106802_19-05-2025_11_02_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23106802 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23106802 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23110969 order by id desc limit 1 Qualite : 0.10507966553069274 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23110971_19-05-2025_12_21_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23110971 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23110971 AND mptpi.`type`=3594 To do Qualite : 0.18118987911522633 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118015 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118015 AND mptpi.`type`=3594 To do Qualite : 0.13490794676941195 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118018_19-05-2025_15_04_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118018 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118018 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23127824 order by id desc limit 1 Qualite : 0.1319593770667989 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23127827_19-05-2025_17_55_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23127827 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23127827 AND mptpi.`type`=3594 To do Qualite : 0.18142057442370754 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23122127_19-05-2025_16_24_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23122127 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23122127 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23122145 order by id desc limit 1 Qualite : 0.13344639331427013 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23124150_19-05-2025_16_51_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23124150 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23124150 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23127829 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23133787 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23131825 order by id desc limit 1 Qualite : 0.12525922610435958 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23133812_19-05-2025_19_46_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23133812 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23133812 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'19052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359488841, 1359488808, 1359488758, 1359488721, 1359488700, 1359488636, 1359487983, 1359487958] Looping around the photos to save general results len do output : 1 /23133812Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488841', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488808', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488758', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488721', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488700', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359488636', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487983', None, None, None, None, None, '2898167') ('3318', None, None, None, None, None, None, None, '2898167') ('3318', '23133812', '1359487958', None, None, None, None, None, '2898167') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013622283935546875 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.173747777938843 time spend to save output : 0.01389002799987793 total time spend for step 10 : 7.187637805938721 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 141.93user 161.83system 6:12.93elapsed 81%CPU (0avgtext+0avgdata 4351024maxresident)k 501840inputs+144424outputs (8386major+14366650minor)pagefaults 0swaps