TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2797056'] with mtr_portfolio_ids : ['22712582'] and first list_photo_ids : [] new path : /proc/317658/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.2465910911560059 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue May 6 10:20:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 10:20:33.380759: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 10:20:33.411248: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 10:20:33.413454: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2080000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 10:20:33.413483: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 10:20:33.417468: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 10:20:33.690170: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x207061b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 10:20:33.690242: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 10:20:33.691815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 10:20:33.692278: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 10:20:33.695449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 10:20:33.698600: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 10:20:33.699218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 10:20:33.701769: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 10:20:33.702918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 10:20:33.709886: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 10:20:33.713420: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 10:20:33.713532: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 10:20:33.715373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 10:20:33.715420: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 10:20:33.715436: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 10:20:33.728051: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10013 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 10:20:34.054851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 10:20:34.054987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 10:20:34.055010: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 10:20:34.055027: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 10:20:34.055043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 10:20:34.055059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 10:20:34.055075: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 10:20:34.055091: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 10:20:34.056277: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 10:20:34.057407: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 10:20:34.057442: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 10:20:34.057464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 10:20:34.057480: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 10:20:34.057496: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 10:20:34.057512: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 10:20:34.057527: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 10:20:34.057543: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 10:20:34.058723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 10:20:34.058756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 10:20:34.058764: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 10:20:34.058771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 10:20:34.060090: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10013 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 10:20:46.193836: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 10:20:46.369483: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 40 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 45 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 40 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 3.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 35 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 44 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 Detection mask done ! Trying to reset tf kernel 318349 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 555 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5866 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.003582000732421875 nb_pixel_total : 61366 time to create 1 rle with old method : 0.07126784324645996 length of segment : 468 time for calcul the mask position with numpy : 0.0010957717895507812 nb_pixel_total : 29219 time to create 1 rle with old method : 0.03349137306213379 length of segment : 198 time for calcul the mask position with numpy : 0.0059893131256103516 nb_pixel_total : 163259 time to create 1 rle with new method : 0.009792804718017578 length of segment : 437 time for calcul the mask position with numpy : 0.0051610469818115234 nb_pixel_total : 98413 time to create 1 rle with old method : 0.128859281539917 length of segment : 344 time for calcul the mask position with numpy : 0.004015207290649414 nb_pixel_total : 56696 time to create 1 rle with old method : 0.06475281715393066 length of segment : 355 time for calcul the mask position with numpy : 0.0005784034729003906 nb_pixel_total : 33628 time to create 1 rle with old method : 0.038873910903930664 length of segment : 282 time for calcul the mask position with numpy : 0.15022587776184082 nb_pixel_total : 280446 time to create 1 rle with new method : 0.04364418983459473 length of segment : 539 time for calcul the mask position with numpy : 0.0013051033020019531 nb_pixel_total : 34679 time to create 1 rle with old method : 0.03868913650512695 length of segment : 331 time for calcul the mask position with numpy : 0.0015032291412353516 nb_pixel_total : 26720 time to create 1 rle with old method : 0.031047344207763672 length of segment : 208 time for calcul the mask position with numpy : 0.009934663772583008 nb_pixel_total : 130955 time to create 1 rle with old method : 0.15360140800476074 length of segment : 726 time for calcul the mask position with numpy : 0.0018885135650634766 nb_pixel_total : 52245 time to create 1 rle with old method : 0.08011245727539062 length of segment : 319 time for calcul the mask position with numpy : 0.003661632537841797 nb_pixel_total : 67374 time to create 1 rle with old method : 0.07905340194702148 length of segment : 325 time for calcul the mask position with numpy : 0.0008027553558349609 nb_pixel_total : 13281 time to create 1 rle with old method : 0.01553487777709961 length of segment : 215 time for calcul the mask position with numpy : 0.0012214183807373047 nb_pixel_total : 25793 time to create 1 rle with old method : 0.030557870864868164 length of segment : 211 time for calcul the mask position with numpy : 0.005819082260131836 nb_pixel_total : 128954 time to create 1 rle with old method : 0.15144729614257812 length of segment : 525 time for calcul the mask position with numpy : 0.0004563331604003906 nb_pixel_total : 12087 time to create 1 rle with old method : 0.014374256134033203 length of segment : 152 time for calcul the mask position with numpy : 0.0005486011505126953 nb_pixel_total : 11054 time to create 1 rle with old method : 0.013478517532348633 length of segment : 114 time for calcul the mask position with numpy : 0.011637687683105469 nb_pixel_total : 224975 time to create 1 rle with new method : 0.024454116821289062 length of segment : 519 time for calcul the mask position with numpy : 0.0008261203765869141 nb_pixel_total : 12979 time to create 1 rle with old method : 0.01524806022644043 length of segment : 119 time for calcul the mask position with numpy : 0.0064144134521484375 nb_pixel_total : 156215 time to create 1 rle with new method : 0.01730656623840332 length of segment : 501 time for calcul the mask position with numpy : 0.0007200241088867188 nb_pixel_total : 13677 time to create 1 rle with old method : 0.016434669494628906 length of segment : 145 time for calcul the mask position with numpy : 0.0007736682891845703 nb_pixel_total : 17855 time to create 1 rle with old method : 0.02091383934020996 length of segment : 174 time for calcul the mask position with numpy : 0.0017766952514648438 nb_pixel_total : 36704 time to create 1 rle with old method : 0.04204440116882324 length of segment : 241 time for calcul the mask position with numpy : 0.001664876937866211 nb_pixel_total : 59849 time to create 1 rle with old method : 0.0712885856628418 length of segment : 462 time for calcul the mask position with numpy : 0.0016255378723144531 nb_pixel_total : 37152 time to create 1 rle with old method : 0.04277396202087402 length of segment : 230 time for calcul the mask position with numpy : 0.005084037780761719 nb_pixel_total : 179510 time to create 1 rle with new method : 0.012438535690307617 length of segment : 457 time for calcul the mask position with numpy : 0.004489898681640625 nb_pixel_total : 56789 time to create 1 rle with old method : 0.06599760055541992 length of segment : 303 time for calcul the mask position with numpy : 0.0027556419372558594 nb_pixel_total : 56474 time to create 1 rle with old method : 0.06588315963745117 length of segment : 326 time for calcul the mask position with numpy : 0.0018436908721923828 nb_pixel_total : 57277 time to create 1 rle with old method : 0.06687593460083008 length of segment : 334 time for calcul the mask position with numpy : 0.003900289535522461 nb_pixel_total : 118732 time to create 1 rle with old method : 0.13403725624084473 length of segment : 504 time for calcul the mask position with numpy : 0.0013728141784667969 nb_pixel_total : 45868 time to create 1 rle with old method : 0.05427432060241699 length of segment : 235 time for calcul the mask position with numpy : 0.011314868927001953 nb_pixel_total : 270516 time to create 1 rle with new method : 0.09549427032470703 length of segment : 597 time for calcul the mask position with numpy : 0.004988670349121094 nb_pixel_total : 135196 time to create 1 rle with old method : 0.14900779724121094 length of segment : 734 time for calcul the mask position with numpy : 0.0030105113983154297 nb_pixel_total : 90591 time to create 1 rle with old method : 0.1260535717010498 length of segment : 319 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 10975 time to create 1 rle with old method : 0.022371292114257812 length of segment : 110 time for calcul the mask position with numpy : 0.0007607936859130859 nb_pixel_total : 12466 time to create 1 rle with old method : 0.017374515533447266 length of segment : 170 time for calcul the mask position with numpy : 0.0013878345489501953 nb_pixel_total : 32948 time to create 1 rle with old method : 0.03723883628845215 length of segment : 329 time for calcul the mask position with numpy : 0.0009274482727050781 nb_pixel_total : 18873 time to create 1 rle with old method : 0.02153301239013672 length of segment : 189 time for calcul the mask position with numpy : 0.0013620853424072266 nb_pixel_total : 35585 time to create 1 rle with old method : 0.04117894172668457 length of segment : 206 time for calcul the mask position with numpy : 0.0007562637329101562 nb_pixel_total : 10898 time to create 1 rle with old method : 0.01221466064453125 length of segment : 294 time for calcul the mask position with numpy : 0.0011932849884033203 nb_pixel_total : 20912 time to create 1 rle with old method : 0.023940086364746094 length of segment : 246 time for calcul the mask position with numpy : 0.0005812644958496094 nb_pixel_total : 12088 time to create 1 rle with old method : 0.01406550407409668 length of segment : 191 time for calcul the mask position with numpy : 0.0038628578186035156 nb_pixel_total : 98899 time to create 1 rle with old method : 0.1195211410522461 length of segment : 347 time for calcul the mask position with numpy : 0.0011913776397705078 nb_pixel_total : 32094 time to create 1 rle with old method : 0.04588818550109863 length of segment : 209 time for calcul the mask position with numpy : 0.0018651485443115234 nb_pixel_total : 51520 time to create 1 rle with old method : 0.05987405776977539 length of segment : 205 time for calcul the mask position with numpy : 0.00347900390625 nb_pixel_total : 104614 time to create 1 rle with old method : 0.11454486846923828 length of segment : 393 time for calcul the mask position with numpy : 0.001310110092163086 nb_pixel_total : 31836 time to create 1 rle with old method : 0.03712964057922363 length of segment : 242 time for calcul the mask position with numpy : 0.0015444755554199219 nb_pixel_total : 49039 time to create 1 rle with old method : 0.056664466857910156 length of segment : 294 time for calcul the mask position with numpy : 0.00036787986755371094 nb_pixel_total : 4638 time to create 1 rle with old method : 0.005559444427490234 length of segment : 70 time for calcul the mask position with numpy : 0.0035254955291748047 nb_pixel_total : 88486 time to create 1 rle with old method : 0.10521483421325684 length of segment : 384 time for calcul the mask position with numpy : 0.003789663314819336 nb_pixel_total : 52132 time to create 1 rle with old method : 0.05968976020812988 length of segment : 270 time for calcul the mask position with numpy : 0.005941867828369141 nb_pixel_total : 157608 time to create 1 rle with new method : 0.01288604736328125 length of segment : 537 time for calcul the mask position with numpy : 0.015738964080810547 nb_pixel_total : 414994 time to create 1 rle with new method : 0.1605851650238037 length of segment : 903 time for calcul the mask position with numpy : 0.0006346702575683594 nb_pixel_total : 16933 time to create 1 rle with old method : 0.021530866622924805 length of segment : 94 time for calcul the mask position with numpy : 0.0006246566772460938 nb_pixel_total : 15481 time to create 1 rle with old method : 0.021017074584960938 length of segment : 236 time for calcul the mask position with numpy : 0.009851694107055664 nb_pixel_total : 263149 time to create 1 rle with new method : 0.01979804039001465 length of segment : 663 time for calcul the mask position with numpy : 0.004669189453125 nb_pixel_total : 101839 time to create 1 rle with old method : 0.13244366645812988 length of segment : 520 time for calcul the mask position with numpy : 0.0006444454193115234 nb_pixel_total : 12492 time to create 1 rle with old method : 0.014313697814941406 length of segment : 159 time for calcul the mask position with numpy : 0.0007326602935791016 nb_pixel_total : 8735 time to create 1 rle with old method : 0.015134096145629883 length of segment : 168 time for calcul the mask position with numpy : 0.0011856555938720703 nb_pixel_total : 16358 time to create 1 rle with old method : 0.029891490936279297 length of segment : 164 time for calcul the mask position with numpy : 0.0006060600280761719 nb_pixel_total : 11918 time to create 1 rle with old method : 0.013586044311523438 length of segment : 206 time for calcul the mask position with numpy : 0.004242420196533203 nb_pixel_total : 115151 time to create 1 rle with old method : 0.1297931671142578 length of segment : 420 time for calcul the mask position with numpy : 0.0004944801330566406 nb_pixel_total : 9051 time to create 1 rle with old method : 0.010560274124145508 length of segment : 109 time for calcul the mask position with numpy : 0.0004253387451171875 nb_pixel_total : 8513 time to create 1 rle with old method : 0.009853839874267578 length of segment : 86 time for calcul the mask position with numpy : 0.011194229125976562 nb_pixel_total : 276985 time to create 1 rle with new method : 0.027666568756103516 length of segment : 568 time for calcul the mask position with numpy : 0.0010509490966796875 nb_pixel_total : 55802 time to create 1 rle with old method : 0.062177181243896484 length of segment : 411 time for calcul the mask position with numpy : 0.0017082691192626953 nb_pixel_total : 25630 time to create 1 rle with old method : 0.029941797256469727 length of segment : 214 time for calcul the mask position with numpy : 0.0130767822265625 nb_pixel_total : 296494 time to create 1 rle with new method : 0.12233257293701172 length of segment : 692 time for calcul the mask position with numpy : 0.0012161731719970703 nb_pixel_total : 26815 time to create 1 rle with old method : 0.030925273895263672 length of segment : 176 time for calcul the mask position with numpy : 0.0057985782623291016 nb_pixel_total : 60020 time to create 1 rle with old method : 0.07233119010925293 length of segment : 676 time for calcul the mask position with numpy : 0.0034182071685791016 nb_pixel_total : 107617 time to create 1 rle with old method : 0.14474940299987793 length of segment : 444 time for calcul the mask position with numpy : 0.000217437744140625 nb_pixel_total : 3959 time to create 1 rle with old method : 0.005475521087646484 length of segment : 71 time for calcul the mask position with numpy : 0.0016846656799316406 nb_pixel_total : 47610 time to create 1 rle with old method : 0.054821014404296875 length of segment : 344 time for calcul the mask position with numpy : 0.004194021224975586 nb_pixel_total : 106590 time to create 1 rle with old method : 0.14243435859680176 length of segment : 324 time for calcul the mask position with numpy : 0.0018150806427001953 nb_pixel_total : 39370 time to create 1 rle with old method : 0.05380105972290039 length of segment : 235 time for calcul the mask position with numpy : 0.0009229183197021484 nb_pixel_total : 24454 time to create 1 rle with old method : 0.028374433517456055 length of segment : 203 time for calcul the mask position with numpy : 0.0006830692291259766 nb_pixel_total : 25353 time to create 1 rle with old method : 0.030330896377563477 length of segment : 221 time for calcul the mask position with numpy : 0.0009930133819580078 nb_pixel_total : 22013 time to create 1 rle with old method : 0.026193618774414062 length of segment : 197 time for calcul the mask position with numpy : 0.00022292137145996094 nb_pixel_total : 9956 time to create 1 rle with old method : 0.01242828369140625 length of segment : 79 time for calcul the mask position with numpy : 0.002651214599609375 nb_pixel_total : 55809 time to create 1 rle with old method : 0.06372237205505371 length of segment : 338 time for calcul the mask position with numpy : 0.001796722412109375 nb_pixel_total : 52175 time to create 1 rle with old method : 0.06822323799133301 length of segment : 312 time for calcul the mask position with numpy : 0.0014061927795410156 nb_pixel_total : 4182 time to create 1 rle with old method : 0.005297422409057617 length of segment : 244 time for calcul the mask position with numpy : 0.0017430782318115234 nb_pixel_total : 37901 time to create 1 rle with old method : 0.049036264419555664 length of segment : 146 time for calcul the mask position with numpy : 0.002919435501098633 nb_pixel_total : 92757 time to create 1 rle with old method : 0.12374472618103027 length of segment : 345 time for calcul the mask position with numpy : 0.0006308555603027344 nb_pixel_total : 14040 time to create 1 rle with old method : 0.01660633087158203 length of segment : 124 time for calcul the mask position with numpy : 0.00034689903259277344 nb_pixel_total : 6776 time to create 1 rle with old method : 0.008146286010742188 length of segment : 106 time for calcul the mask position with numpy : 0.0023529529571533203 nb_pixel_total : 41050 time to create 1 rle with old method : 0.04683518409729004 length of segment : 382 time for calcul the mask position with numpy : 0.00040411949157714844 nb_pixel_total : 9615 time to create 1 rle with old method : 0.011363983154296875 length of segment : 220 time for calcul the mask position with numpy : 0.0014035701751708984 nb_pixel_total : 34116 time to create 1 rle with old method : 0.0389096736907959 length of segment : 272 time for calcul the mask position with numpy : 0.0011696815490722656 nb_pixel_total : 28489 time to create 1 rle with old method : 0.03331136703491211 length of segment : 232 time for calcul the mask position with numpy : 0.001234292984008789 nb_pixel_total : 27076 time to create 1 rle with old method : 0.031852006912231445 length of segment : 206 time for calcul the mask position with numpy : 0.0009431838989257812 nb_pixel_total : 15748 time to create 1 rle with old method : 0.019454479217529297 length of segment : 129 time for calcul the mask position with numpy : 0.0015995502471923828 nb_pixel_total : 30849 time to create 1 rle with old method : 0.036284446716308594 length of segment : 266 time for calcul the mask position with numpy : 0.0009663105010986328 nb_pixel_total : 8045 time to create 1 rle with old method : 0.011521577835083008 length of segment : 140 time for calcul the mask position with numpy : 0.006343364715576172 nb_pixel_total : 118889 time to create 1 rle with old method : 0.13419413566589355 length of segment : 546 time for calcul the mask position with numpy : 0.0005893707275390625 nb_pixel_total : 13582 time to create 1 rle with old method : 0.016176700592041016 length of segment : 115 time for calcul the mask position with numpy : 0.0012192726135253906 nb_pixel_total : 27236 time to create 1 rle with old method : 0.03206658363342285 length of segment : 168 time for calcul the mask position with numpy : 0.0018010139465332031 nb_pixel_total : 39842 time to create 1 rle with old method : 0.04575490951538086 length of segment : 333 time for calcul the mask position with numpy : 0.00156402587890625 nb_pixel_total : 23512 time to create 1 rle with old method : 0.02867269515991211 length of segment : 158 time for calcul the mask position with numpy : 0.0011470317840576172 nb_pixel_total : 25896 time to create 1 rle with old method : 0.033089399337768555 length of segment : 207 time for calcul the mask position with numpy : 0.0017442703247070312 nb_pixel_total : 36362 time to create 1 rle with old method : 0.050989627838134766 length of segment : 321 time for calcul the mask position with numpy : 0.0009579658508300781 nb_pixel_total : 21818 time to create 1 rle with old method : 0.028580427169799805 length of segment : 131 time for calcul the mask position with numpy : 0.00421905517578125 nb_pixel_total : 92216 time to create 1 rle with old method : 0.11253833770751953 length of segment : 407 time for calcul the mask position with numpy : 0.002447366714477539 nb_pixel_total : 40268 time to create 1 rle with old method : 0.06076860427856445 length of segment : 254 time for calcul the mask position with numpy : 0.000911712646484375 nb_pixel_total : 13840 time to create 1 rle with old method : 0.017142772674560547 length of segment : 129 time for calcul the mask position with numpy : 0.0016105175018310547 nb_pixel_total : 26488 time to create 1 rle with old method : 0.03351926803588867 length of segment : 255 time for calcul the mask position with numpy : 0.0008039474487304688 nb_pixel_total : 18836 time to create 1 rle with old method : 0.021297454833984375 length of segment : 163 time for calcul the mask position with numpy : 0.0008072853088378906 nb_pixel_total : 20739 time to create 1 rle with old method : 0.023906230926513672 length of segment : 181 time for calcul the mask position with numpy : 0.0006875991821289062 nb_pixel_total : 16937 time to create 1 rle with old method : 0.019637346267700195 length of segment : 180 time for calcul the mask position with numpy : 0.0006787776947021484 nb_pixel_total : 13283 time to create 1 rle with old method : 0.01542973518371582 length of segment : 150 time for calcul the mask position with numpy : 0.0011599063873291016 nb_pixel_total : 7342 time to create 1 rle with old method : 0.01004648208618164 length of segment : 104 time for calcul the mask position with numpy : 0.0009171962738037109 nb_pixel_total : 22053 time to create 1 rle with old method : 0.0270383358001709 length of segment : 113 time for calcul the mask position with numpy : 0.003423929214477539 nb_pixel_total : 56756 time to create 1 rle with old method : 0.06696295738220215 length of segment : 320 time for calcul the mask position with numpy : 0.0005712509155273438 nb_pixel_total : 11850 time to create 1 rle with old method : 0.015570878982543945 length of segment : 98 time for calcul the mask position with numpy : 0.0009930133819580078 nb_pixel_total : 27516 time to create 1 rle with old method : 0.03129076957702637 length of segment : 197 time for calcul the mask position with numpy : 0.003172636032104492 nb_pixel_total : 24897 time to create 1 rle with old method : 0.028920650482177734 length of segment : 393 time for calcul the mask position with numpy : 0.00405573844909668 nb_pixel_total : 69875 time to create 1 rle with old method : 0.07615494728088379 length of segment : 333 time for calcul the mask position with numpy : 0.0026726722717285156 nb_pixel_total : 87262 time to create 1 rle with old method : 0.09780120849609375 length of segment : 307 time for calcul the mask position with numpy : 0.0010302066802978516 nb_pixel_total : 28682 time to create 1 rle with old method : 0.03322434425354004 length of segment : 225 time for calcul the mask position with numpy : 0.002206563949584961 nb_pixel_total : 30506 time to create 1 rle with old method : 0.0381619930267334 length of segment : 284 time for calcul the mask position with numpy : 0.0009512901306152344 nb_pixel_total : 25060 time to create 1 rle with old method : 0.02910447120666504 length of segment : 174 time for calcul the mask position with numpy : 0.0011179447174072266 nb_pixel_total : 25436 time to create 1 rle with old method : 0.030128002166748047 length of segment : 180 time for calcul the mask position with numpy : 0.0007085800170898438 nb_pixel_total : 16190 time to create 1 rle with old method : 0.024629592895507812 length of segment : 117 time for calcul the mask position with numpy : 0.0018317699432373047 nb_pixel_total : 32451 time to create 1 rle with old method : 0.049391746520996094 length of segment : 288 time for calcul the mask position with numpy : 0.0023584365844726562 nb_pixel_total : 65884 time to create 1 rle with old method : 0.09529805183410645 length of segment : 365 time for calcul the mask position with numpy : 0.0024232864379882812 nb_pixel_total : 87514 time to create 1 rle with old method : 0.10080170631408691 length of segment : 356 time for calcul the mask position with numpy : 0.0011949539184570312 nb_pixel_total : 36287 time to create 1 rle with old method : 0.044689178466796875 length of segment : 230 time for calcul the mask position with numpy : 0.00075531005859375 nb_pixel_total : 17108 time to create 1 rle with old method : 0.020245790481567383 length of segment : 127 time for calcul the mask position with numpy : 0.0013818740844726562 nb_pixel_total : 32692 time to create 1 rle with old method : 0.03761005401611328 length of segment : 319 time for calcul the mask position with numpy : 0.008902549743652344 nb_pixel_total : 255521 time to create 1 rle with new method : 0.01828908920288086 length of segment : 651 time for calcul the mask position with numpy : 0.0020627975463867188 nb_pixel_total : 56468 time to create 1 rle with old method : 0.0648047924041748 length of segment : 238 time for calcul the mask position with numpy : 0.0007214546203613281 nb_pixel_total : 22115 time to create 1 rle with old method : 0.027201414108276367 length of segment : 111 time for calcul the mask position with numpy : 0.0009300708770751953 nb_pixel_total : 20710 time to create 1 rle with old method : 0.035092830657958984 length of segment : 171 time spent for convertir_results : 14.416846513748169 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 133 chid ids of type : 3594 Number RLEs to save : 38079 save missing photos in datou_result : time spend for datou_step_exec : 73.10384845733643 time spend to save output : 2.3938372135162354 total time spend for step 1 : 75.49768567085266 step2:crop_condition Tue May 6 10:21:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 133 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 80 About to insert : list_path_to_insert length 80 new photo from crops ! About to upload 80 photos upload in portfolio : 3736932 init cache_photo without model_param we have 80 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746519726_317658 we have uploaded 80 photos in the portfolio 3736932 time of upload the photos Elapsed time : 19.474087715148926 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! About to upload 22 photos upload in portfolio : 3736932 init cache_photo without model_param we have 22 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746519753_317658 we have uploaded 22 photos in the portfolio 3736932 time of upload the photos Elapsed time : 6.1286725997924805 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746519761_317658 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6575701236724854 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 28 About to insert : list_path_to_insert length 28 new photo from crops ! About to upload 28 photos upload in portfolio : 3736932 init cache_photo without model_param we have 28 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746519773_317658 we have uploaded 28 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.415789604187012 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746519781_317658 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7342379093170166 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356308633, 1356308630, 1356308628, 1356308580, 1356308576, 1356308571, 1356308566, 1356308531] Looping around the photos to save general results len do output : 133 /1356321762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356321989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308633', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308630', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308628', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308580', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308576', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308571', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308566', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308531', None, None, None, None, None, '2797056') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 407 time used for this insertion : 0.06969189643859863 save_final save missing photos in datou_result : time spend for datou_step_exec : 76.33258843421936 time spend to save output : 0.0772850513458252 total time spend for step 2 : 76.40987348556519 step3:rle_unique_nms_with_priority Tue May 6 10:23:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 133 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 23 nb_hashtags : 3 time to prepare the origin masks : 6.930290222167969 time for calcul the mask position with numpy : 0.4112420082092285 nb_pixel_total : 5365548 time to create 1 rle with new method : 0.6781699657440186 time for calcul the mask position with numpy : 0.02081918716430664 nb_pixel_total : 36704 time to create 1 rle with old method : 0.03930497169494629 time for calcul the mask position with numpy : 0.020128250122070312 nb_pixel_total : 17855 time to create 1 rle with old method : 0.019287824630737305 time for calcul the mask position with numpy : 0.020252466201782227 nb_pixel_total : 13677 time to create 1 rle with old method : 0.014574527740478516 time for calcul the mask position with numpy : 0.021160364151000977 nb_pixel_total : 156205 time to create 1 rle with new method : 0.5531730651855469 time for calcul the mask position with numpy : 0.021312713623046875 nb_pixel_total : 12979 time to create 1 rle with old method : 0.014613866806030273 time for calcul the mask position with numpy : 0.023311853408813477 nb_pixel_total : 224975 time to create 1 rle with new method : 0.444155216217041 time for calcul the mask position with numpy : 0.02071213722229004 nb_pixel_total : 11054 time to create 1 rle with old method : 0.01158285140991211 time for calcul the mask position with numpy : 0.020769119262695312 nb_pixel_total : 12087 time to create 1 rle with old method : 0.013067245483398438 time for calcul the mask position with numpy : 0.021246671676635742 nb_pixel_total : 128954 time to create 1 rle with old method : 0.13875293731689453 time for calcul the mask position with numpy : 0.023051738739013672 nb_pixel_total : 25793 time to create 1 rle with old method : 0.028994083404541016 time for calcul the mask position with numpy : 0.03479170799255371 nb_pixel_total : 13281 time to create 1 rle with old method : 0.015889406204223633 time for calcul the mask position with numpy : 0.03487277030944824 nb_pixel_total : 67374 time to create 1 rle with old method : 0.07860279083251953 time for calcul the mask position with numpy : 0.03409886360168457 nb_pixel_total : 52245 time to create 1 rle with old method : 0.057240962982177734 time for calcul the mask position with numpy : 0.03380250930786133 nb_pixel_total : 130955 time to create 1 rle with old method : 0.14608526229858398 time for calcul the mask position with numpy : 0.03399801254272461 nb_pixel_total : 26720 time to create 1 rle with old method : 0.033946990966796875 time for calcul the mask position with numpy : 0.03412795066833496 nb_pixel_total : 34679 time to create 1 rle with old method : 0.03853034973144531 time for calcul the mask position with numpy : 0.03374147415161133 nb_pixel_total : 280446 time to create 1 rle with new method : 2.0419108867645264 time for calcul the mask position with numpy : 0.0247952938079834 nb_pixel_total : 29756 time to create 1 rle with old method : 0.03327679634094238 time for calcul the mask position with numpy : 0.022416353225708008 nb_pixel_total : 56696 time to create 1 rle with old method : 0.0638425350189209 time for calcul the mask position with numpy : 0.02255105972290039 nb_pixel_total : 98413 time to create 1 rle with old method : 0.10961604118347168 time for calcul the mask position with numpy : 0.023102998733520508 nb_pixel_total : 163259 time to create 1 rle with new method : 0.9953467845916748 time for calcul the mask position with numpy : 0.022241592407226562 nb_pixel_total : 29219 time to create 1 rle with old method : 0.03285098075866699 time for calcul the mask position with numpy : 0.022390365600585938 nb_pixel_total : 61366 time to create 1 rle with old method : 0.06834220886230469 create new chi : 6.8015217781066895 time to delete rle : 0.02356100082397461 batch 1 Loaded 47 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++Number RLEs to save : 16873 TO DO : save crop sub photo not yet done ! save time : 1.1829533576965332 nb_obj : 20 nb_hashtags : 4 time to prepare the origin masks : 8.239278554916382 time for calcul the mask position with numpy : 0.29929065704345703 nb_pixel_total : 5694510 time to create 1 rle with new method : 0.6706821918487549 time for calcul the mask position with numpy : 0.022139787673950195 nb_pixel_total : 98899 time to create 1 rle with old method : 0.139054536819458 time for calcul the mask position with numpy : 0.020641565322875977 nb_pixel_total : 12088 time to create 1 rle with old method : 0.013348579406738281 time for calcul the mask position with numpy : 0.0206148624420166 nb_pixel_total : 20912 time to create 1 rle with old method : 0.023046016693115234 time for calcul the mask position with numpy : 0.021114826202392578 nb_pixel_total : 10898 time to create 1 rle with old method : 0.012095212936401367 time for calcul the mask position with numpy : 0.021841764450073242 nb_pixel_total : 33163 time to create 1 rle with old method : 0.036661624908447266 time for calcul the mask position with numpy : 0.02123546600341797 nb_pixel_total : 18873 time to create 1 rle with old method : 0.021213293075561523 time for calcul the mask position with numpy : 0.021131277084350586 nb_pixel_total : 32948 time to create 1 rle with old method : 0.0359644889831543 time for calcul the mask position with numpy : 0.021520614624023438 nb_pixel_total : 9020 time to create 1 rle with old method : 0.009854316711425781 time for calcul the mask position with numpy : 0.021679401397705078 nb_pixel_total : 10975 time to create 1 rle with old method : 0.012128114700317383 time for calcul the mask position with numpy : 0.02177715301513672 nb_pixel_total : 90591 time to create 1 rle with old method : 0.09755969047546387 time for calcul the mask position with numpy : 0.02075052261352539 nb_pixel_total : 135196 time to create 1 rle with old method : 0.14819765090942383 time for calcul the mask position with numpy : 0.02442312240600586 nb_pixel_total : 270516 time to create 1 rle with new method : 0.39968037605285645 time for calcul the mask position with numpy : 0.02194666862487793 nb_pixel_total : 45868 time to create 1 rle with old method : 0.050264596939086914 time for calcul the mask position with numpy : 0.022069215774536133 nb_pixel_total : 118732 time to create 1 rle with old method : 0.13575506210327148 time for calcul the mask position with numpy : 0.02133011817932129 nb_pixel_total : 57277 time to create 1 rle with old method : 0.06307482719421387 time for calcul the mask position with numpy : 0.02214193344116211 nb_pixel_total : 56474 time to create 1 rle with old method : 0.06197690963745117 time for calcul the mask position with numpy : 0.021533727645874023 nb_pixel_total : 56789 time to create 1 rle with old method : 0.06252598762512207 time for calcul the mask position with numpy : 0.022044897079467773 nb_pixel_total : 179510 time to create 1 rle with new method : 0.7155790328979492 time for calcul the mask position with numpy : 0.021676063537597656 nb_pixel_total : 37152 time to create 1 rle with old method : 0.039453744888305664 time for calcul the mask position with numpy : 0.021406888961791992 nb_pixel_total : 59849 time to create 1 rle with old method : 0.06379127502441406 create new chi : 3.63881778717041 time to delete rle : 0.0018339157104492188 batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++++++++Number RLEs to save : 15057 TO DO : save crop sub photo not yet done ! save time : 0.9493227005004883 nb_obj : 21 nb_hashtags : 3 time to prepare the origin masks : 7.164471864700317 time for calcul the mask position with numpy : 0.36884164810180664 nb_pixel_total : 5483659 time to create 1 rle with new method : 0.8963415622711182 time for calcul the mask position with numpy : 0.03277873992919922 nb_pixel_total : 8513 time to create 1 rle with old method : 0.009348630905151367 time for calcul the mask position with numpy : 0.020866870880126953 nb_pixel_total : 9051 time to create 1 rle with old method : 0.009922027587890625 time for calcul the mask position with numpy : 0.021430492401123047 nb_pixel_total : 115151 time to create 1 rle with old method : 0.12694644927978516 time for calcul the mask position with numpy : 0.020643234252929688 nb_pixel_total : 11918 time to create 1 rle with old method : 0.01277613639831543 time for calcul the mask position with numpy : 0.02123880386352539 nb_pixel_total : 16358 time to create 1 rle with old method : 0.018119335174560547 time for calcul the mask position with numpy : 0.022144556045532227 nb_pixel_total : 8735 time to create 1 rle with old method : 0.009857892990112305 time for calcul the mask position with numpy : 0.02156829833984375 nb_pixel_total : 12492 time to create 1 rle with old method : 0.014012336730957031 time for calcul the mask position with numpy : 0.02248406410217285 nb_pixel_total : 101839 time to create 1 rle with old method : 0.11345791816711426 time for calcul the mask position with numpy : 0.022290468215942383 nb_pixel_total : 263149 time to create 1 rle with new method : 0.679069995880127 time for calcul the mask position with numpy : 0.0214691162109375 nb_pixel_total : 15481 time to create 1 rle with old method : 0.016927719116210938 time for calcul the mask position with numpy : 0.021529197692871094 nb_pixel_total : 16933 time to create 1 rle with old method : 0.018758296966552734 time for calcul the mask position with numpy : 0.025632143020629883 nb_pixel_total : 414994 time to create 1 rle with new method : 0.8113176822662354 time for calcul the mask position with numpy : 0.022179841995239258 nb_pixel_total : 157608 time to create 1 rle with new method : 0.5870211124420166 time for calcul the mask position with numpy : 0.023941993713378906 nb_pixel_total : 52132 time to create 1 rle with old method : 0.05633997917175293 time for calcul the mask position with numpy : 0.021938800811767578 nb_pixel_total : 88486 time to create 1 rle with old method : 0.09665632247924805 time for calcul the mask position with numpy : 0.022205352783203125 nb_pixel_total : 4638 time to create 1 rle with old method : 0.005181789398193359 time for calcul the mask position with numpy : 0.022815465927124023 nb_pixel_total : 49039 time to create 1 rle with old method : 0.054558515548706055 time for calcul the mask position with numpy : 0.027324438095092773 nb_pixel_total : 31836 time to create 1 rle with old method : 0.0356752872467041 time for calcul the mask position with numpy : 0.02173900604248047 nb_pixel_total : 104614 time to create 1 rle with old method : 0.1138315200805664 time for calcul the mask position with numpy : 0.02214503288269043 nb_pixel_total : 51520 time to create 1 rle with old method : 0.05636143684387207 time for calcul the mask position with numpy : 0.02158665657043457 nb_pixel_total : 32094 time to create 1 rle with old method : 0.03565168380737305 create new chi : 4.731565475463867 time to delete rle : 0.0019097328186035156 batch 1 Loaded 43 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++Number RLEs to save : 14824 TO DO : save crop sub photo not yet done ! save time : 0.9374823570251465 nb_obj : 18 nb_hashtags : 3 time to prepare the origin masks : 6.577229738235474 time for calcul the mask position with numpy : 0.6653769016265869 nb_pixel_total : 5888038 time to create 1 rle with new method : 0.8207342624664307 time for calcul the mask position with numpy : 0.037346601486206055 nb_pixel_total : 3577 time to create 1 rle with old method : 0.004185676574707031 time for calcul the mask position with numpy : 0.03773093223571777 nb_pixel_total : 52175 time to create 1 rle with old method : 0.058119773864746094 time for calcul the mask position with numpy : 0.037109375 nb_pixel_total : 55809 time to create 1 rle with old method : 0.062133073806762695 time for calcul the mask position with numpy : 0.034661054611206055 nb_pixel_total : 1627 time to create 1 rle with old method : 0.0019884109497070312 time for calcul the mask position with numpy : 0.04203319549560547 nb_pixel_total : 22013 time to create 1 rle with old method : 0.02828836441040039 time for calcul the mask position with numpy : 0.024831533432006836 nb_pixel_total : 8276 time to create 1 rle with old method : 0.013809919357299805 time for calcul the mask position with numpy : 0.02424025535583496 nb_pixel_total : 24454 time to create 1 rle with old method : 0.031420230865478516 time for calcul the mask position with numpy : 0.023175477981567383 nb_pixel_total : 39370 time to create 1 rle with old method : 0.043627262115478516 time for calcul the mask position with numpy : 0.021778345108032227 nb_pixel_total : 106590 time to create 1 rle with old method : 0.11738419532775879 time for calcul the mask position with numpy : 0.02165842056274414 nb_pixel_total : 47610 time to create 1 rle with old method : 0.05176496505737305 time for calcul the mask position with numpy : 0.021073341369628906 nb_pixel_total : 3704 time to create 1 rle with old method : 0.004203081130981445 time for calcul the mask position with numpy : 0.02808856964111328 nb_pixel_total : 107617 time to create 1 rle with old method : 0.14907026290893555 time for calcul the mask position with numpy : 0.023458242416381836 nb_pixel_total : 60020 time to create 1 rle with old method : 0.09393501281738281 time for calcul the mask position with numpy : 0.022458314895629883 nb_pixel_total : 26815 time to create 1 rle with old method : 0.0296170711517334 time for calcul the mask position with numpy : 0.02506279945373535 nb_pixel_total : 296494 time to create 1 rle with new method : 0.6635820865631104 time for calcul the mask position with numpy : 0.022431135177612305 nb_pixel_total : 25630 time to create 1 rle with old method : 0.028746843338012695 time for calcul the mask position with numpy : 0.022425413131713867 nb_pixel_total : 3436 time to create 1 rle with old method : 0.00418400764465332 time for calcul the mask position with numpy : 0.02405405044555664 nb_pixel_total : 276985 time to create 1 rle with new method : 0.49936699867248535 create new chi : 3.9551331996917725 time to delete rle : 0.002992868423461914 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 12629 TO DO : save crop sub photo not yet done ! save time : 0.8327910900115967 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 4.685242176055908 time for calcul the mask position with numpy : 0.46832871437072754 nb_pixel_total : 6573379 time to create 1 rle with new method : 0.8687143325805664 time for calcul the mask position with numpy : 0.03478121757507324 nb_pixel_total : 13582 time to create 1 rle with old method : 0.015277385711669922 time for calcul the mask position with numpy : 0.03625345230102539 nb_pixel_total : 118889 time to create 1 rle with old method : 0.15540456771850586 time for calcul the mask position with numpy : 0.03852486610412598 nb_pixel_total : 8045 time to create 1 rle with old method : 0.009198665618896484 time for calcul the mask position with numpy : 0.03569459915161133 nb_pixel_total : 30849 time to create 1 rle with old method : 0.03499770164489746 time for calcul the mask position with numpy : 0.025406599044799805 nb_pixel_total : 15748 time to create 1 rle with old method : 0.017717361450195312 time for calcul the mask position with numpy : 0.025091886520385742 nb_pixel_total : 27076 time to create 1 rle with old method : 0.030216217041015625 time for calcul the mask position with numpy : 0.022618532180786133 nb_pixel_total : 28489 time to create 1 rle with old method : 0.03199028968811035 time for calcul the mask position with numpy : 0.02172565460205078 nb_pixel_total : 34116 time to create 1 rle with old method : 0.03897881507873535 time for calcul the mask position with numpy : 0.02150869369506836 nb_pixel_total : 7543 time to create 1 rle with old method : 0.008586883544921875 time for calcul the mask position with numpy : 0.021404266357421875 nb_pixel_total : 41050 time to create 1 rle with old method : 0.045270442962646484 time for calcul the mask position with numpy : 0.02152705192565918 nb_pixel_total : 6776 time to create 1 rle with old method : 0.007618427276611328 time for calcul the mask position with numpy : 0.021648645401000977 nb_pixel_total : 14040 time to create 1 rle with old method : 0.015619277954101562 time for calcul the mask position with numpy : 0.021692276000976562 nb_pixel_total : 92757 time to create 1 rle with old method : 0.10101056098937988 time for calcul the mask position with numpy : 0.020807266235351562 nb_pixel_total : 37901 time to create 1 rle with old method : 0.04161429405212402 create new chi : 2.2973713874816895 time to delete rle : 0.0012073516845703125 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++++++++Number RLEs to save : 8424 TO DO : save crop sub photo not yet done ! save time : 0.5531539916992188 nb_obj : 21 nb_hashtags : 3 time to prepare the origin masks : 7.457420587539673 time for calcul the mask position with numpy : 0.3676016330718994 nb_pixel_total : 6412843 time to create 1 rle with new method : 1.0101990699768066 time for calcul the mask position with numpy : 0.021344423294067383 nb_pixel_total : 69875 time to create 1 rle with old method : 0.07645153999328613 time for calcul the mask position with numpy : 0.020041227340698242 nb_pixel_total : 24897 time to create 1 rle with old method : 0.0267333984375 time for calcul the mask position with numpy : 0.021635770797729492 nb_pixel_total : 27351 time to create 1 rle with old method : 0.029962539672851562 time for calcul the mask position with numpy : 0.02097797393798828 nb_pixel_total : 11850 time to create 1 rle with old method : 0.012996912002563477 time for calcul the mask position with numpy : 0.02172231674194336 nb_pixel_total : 56756 time to create 1 rle with old method : 0.06224226951599121 time for calcul the mask position with numpy : 0.021537065505981445 nb_pixel_total : 22053 time to create 1 rle with old method : 0.02431797981262207 time for calcul the mask position with numpy : 0.021279096603393555 nb_pixel_total : 7342 time to create 1 rle with old method : 0.008353471755981445 time for calcul the mask position with numpy : 0.023071765899658203 nb_pixel_total : 13283 time to create 1 rle with old method : 0.014525175094604492 time for calcul the mask position with numpy : 0.020499229431152344 nb_pixel_total : 16937 time to create 1 rle with old method : 0.017696142196655273 time for calcul the mask position with numpy : 0.020258665084838867 nb_pixel_total : 20739 time to create 1 rle with old method : 0.022137165069580078 time for calcul the mask position with numpy : 0.020501375198364258 nb_pixel_total : 18836 time to create 1 rle with old method : 0.020389080047607422 time for calcul the mask position with numpy : 0.020832061767578125 nb_pixel_total : 26488 time to create 1 rle with old method : 0.028196096420288086 time for calcul the mask position with numpy : 0.020351409912109375 nb_pixel_total : 13840 time to create 1 rle with old method : 0.01507878303527832 time for calcul the mask position with numpy : 0.02019810676574707 nb_pixel_total : 40268 time to create 1 rle with old method : 0.04298233985900879 time for calcul the mask position with numpy : 0.021023035049438477 nb_pixel_total : 92216 time to create 1 rle with old method : 0.09906768798828125 time for calcul the mask position with numpy : 0.023141860961914062 nb_pixel_total : 21818 time to create 1 rle with old method : 0.02443099021911621 time for calcul the mask position with numpy : 0.023166894912719727 nb_pixel_total : 36362 time to create 1 rle with old method : 0.04034757614135742 time for calcul the mask position with numpy : 0.020964622497558594 nb_pixel_total : 25896 time to create 1 rle with old method : 0.02829265594482422 time for calcul the mask position with numpy : 0.021321773529052734 nb_pixel_total : 23512 time to create 1 rle with old method : 0.026251792907714844 time for calcul the mask position with numpy : 0.021721363067626953 nb_pixel_total : 39842 time to create 1 rle with old method : 0.04404735565185547 time for calcul the mask position with numpy : 0.021764278411865234 nb_pixel_total : 27236 time to create 1 rle with old method : 0.030382871627807617 create new chi : 2.560109853744507 time to delete rle : 0.001678466796875 batch 1 Loaded 43 chid ids of type : 3594 ++++++++++++++++++++++++++++Number RLEs to save : 11340 TO DO : save crop sub photo not yet done ! save time : 0.7775025367736816 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 3.807271718978882 time for calcul the mask position with numpy : 0.7650737762451172 nb_pixel_total : 6769179 time to create 1 rle with new method : 1.3541734218597412 time for calcul the mask position with numpy : 0.033684492111206055 nb_pixel_total : 35474 time to create 1 rle with old method : 0.0399174690246582 time for calcul the mask position with numpy : 0.03436899185180664 nb_pixel_total : 32451 time to create 1 rle with old method : 0.03589296340942383 time for calcul the mask position with numpy : 0.020977497100830078 nb_pixel_total : 16190 time to create 1 rle with old method : 0.01803278923034668 time for calcul the mask position with numpy : 0.02155780792236328 nb_pixel_total : 25436 time to create 1 rle with old method : 0.027801036834716797 time for calcul the mask position with numpy : 0.021793127059936523 nb_pixel_total : 25060 time to create 1 rle with old method : 0.02748894691467285 time for calcul the mask position with numpy : 0.021555662155151367 nb_pixel_total : 30506 time to create 1 rle with old method : 0.03366851806640625 time for calcul the mask position with numpy : 0.021727323532104492 nb_pixel_total : 28682 time to create 1 rle with old method : 0.032286882400512695 time for calcul the mask position with numpy : 0.021481990814208984 nb_pixel_total : 87262 time to create 1 rle with old method : 0.09651303291320801 create new chi : 2.6649351119995117 time to delete rle : 0.0008862018585205078 batch 1 Loaded 17 chid ids of type : 3594 ++++++++Number RLEs to save : 6003 TO DO : save crop sub photo not yet done ! save time : 0.43880128860473633 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 3.2321557998657227 time for calcul the mask position with numpy : 0.6088638305664062 nb_pixel_total : 6521825 time to create 1 rle with new method : 0.8056387901306152 time for calcul the mask position with numpy : 0.020642518997192383 nb_pixel_total : 20710 time to create 1 rle with old method : 0.022936105728149414 time for calcul the mask position with numpy : 0.020139455795288086 nb_pixel_total : 22115 time to create 1 rle with old method : 0.024343490600585938 time for calcul the mask position with numpy : 0.023984193801879883 nb_pixel_total : 56468 time to create 1 rle with old method : 0.06253862380981445 time for calcul the mask position with numpy : 0.022815942764282227 nb_pixel_total : 255521 time to create 1 rle with new method : 0.887176513671875 time for calcul the mask position with numpy : 0.026261091232299805 nb_pixel_total : 32692 time to create 1 rle with old method : 0.03677845001220703 time for calcul the mask position with numpy : 0.03075885772705078 nb_pixel_total : 17108 time to create 1 rle with old method : 0.019415855407714844 time for calcul the mask position with numpy : 0.026839017868041992 nb_pixel_total : 36287 time to create 1 rle with old method : 0.04132390022277832 time for calcul the mask position with numpy : 0.02849602699279785 nb_pixel_total : 87514 time to create 1 rle with old method : 0.1475529670715332 create new chi : 2.921936273574829 time to delete rle : 0.0019576549530029297 batch 1 Loaded 17 chid ids of type : 3594 +++++++++Number RLEs to save : 6566 TO DO : save crop sub photo not yet done ! save time : 0.48558926582336426 map_output_result : {1356308633: (0.0, 'Should be the crop_list due to order', 0), 1356308630: (0.0, 'Should be the crop_list due to order', 0), 1356308628: (0.0, 'Should be the crop_list due to order', 0), 1356308580: (0.0, 'Should be the crop_list due to order', 0), 1356308576: (0.0, 'Should be the crop_list due to order', 0), 1356308571: (0.0, 'Should be the crop_list due to order', 0), 1356308566: (0.0, 'Should be the crop_list due to order', 0), 1356308531: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356308633, 1356308630, 1356308628, 1356308580, 1356308576, 1356308571, 1356308566, 1356308531] Looping around the photos to save general results len do output : 8 /1356308633.Didn't retrieve data . /1356308630.Didn't retrieve data . /1356308628.Didn't retrieve data . /1356308580.Didn't retrieve data . /1356308576.Didn't retrieve data . /1356308571.Didn't retrieve data . /1356308566.Didn't retrieve data . /1356308531.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308633', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308630', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308628', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308580', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308576', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308571', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308566', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308531', None, None, None, None, None, '2797056') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.016602516174316406 save_final save missing photos in datou_result : time spend for datou_step_exec : 84.90289402008057 time spend to save output : 0.017477035522460938 total time spend for step 3 : 84.92037105560303 step4:ventilate_hashtags_in_portfolio Tue May 6 10:24:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22712582 get user id for portfolio 22712582 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22712582 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','autre','background','pet_fonce','pehd','mal_croppe','metal','environnement','flou','papier','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22712582 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','autre','background','pet_fonce','pehd','mal_croppe','metal','environnement','flou','papier','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22712582 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','autre','background','pet_fonce','pehd','mal_croppe','metal','environnement','flou','papier','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/22712621,22712622,22712623,22712624,22712625,22712626,22712627,22712628,22712629,22712630,22712631?tags=pet_clair,autre,background,pet_fonce,pehd,mal_croppe,metal,environnement,flou,papier,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356308633, 1356308630, 1356308628, 1356308580, 1356308576, 1356308571, 1356308566, 1356308531] Looping around the photos to save general results len do output : 1 /22712582. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308633', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308630', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308628', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308580', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308576', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308571', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308566', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308531', None, None, None, None, None, '2797056') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.014192819595336914 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6425294876098633 time spend to save output : 0.014594793319702148 total time spend for step 4 : 1.6571242809295654 step5:final Tue May 6 10:24:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356308633: ('0.1363949844260621',), 1356308630: ('0.1363949844260621',), 1356308628: ('0.1363949844260621',), 1356308580: ('0.1363949844260621',), 1356308576: ('0.1363949844260621',), 1356308571: ('0.1363949844260621',), 1356308566: ('0.1363949844260621',), 1356308531: ('0.1363949844260621',)} new output for save of step final : {1356308633: ('0.1363949844260621',), 1356308630: ('0.1363949844260621',), 1356308628: ('0.1363949844260621',), 1356308580: ('0.1363949844260621',), 1356308576: ('0.1363949844260621',), 1356308571: ('0.1363949844260621',), 1356308566: ('0.1363949844260621',), 1356308531: ('0.1363949844260621',)} [1356308633, 1356308630, 1356308628, 1356308580, 1356308576, 1356308571, 1356308566, 1356308531] Looping around the photos to save general results len do output : 8 /1356308633.Didn't retrieve data . /1356308630.Didn't retrieve data . /1356308628.Didn't retrieve data . /1356308580.Didn't retrieve data . /1356308576.Didn't retrieve data . /1356308571.Didn't retrieve data . /1356308566.Didn't retrieve data . /1356308531.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308633', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308630', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308628', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308580', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308576', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308571', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308566', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308531', None, None, None, None, None, '2797056') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.016342878341674805 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12591195106506348 time spend to save output : 0.01693439483642578 total time spend for step 5 : 0.14284634590148926 step6:blur_detection Tue May 6 10:24:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111.jpg resize: (2160, 3264) 1356308633 -5.452549631917293 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec.jpg resize: (2160, 3264) 1356308630 -5.342885029736863 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d.jpg resize: (2160, 3264) 1356308628 -1.087307808850906 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6.jpg resize: (2160, 3264) 1356308580 -2.7073552958617118 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63.jpg resize: (2160, 3264) 1356308576 -3.8117594391348697 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766.jpg resize: (2160, 3264) 1356308571 -2.121207087631017 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998.jpg resize: (2160, 3264) 1356308566 -1.955837135530683 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c.jpg resize: (2160, 3264) 1356308531 1.480653038756646 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329180_0.png resize: (329, 515) 1356321762 -2.2387037253544455 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329181_0.png resize: (312, 480) 1356321763 -2.824472156314112 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329178_0.png resize: (178, 247) 1356321764 -1.854317145537729 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329182_0.png resize: (196, 245) 1356321765 -2.050903696761098 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329195_0.png resize: (104, 174) 1356321766 -3.8465436883224573 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329187_0.png resize: (230, 386) 1356321767 -1.811790131965262 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329199_0.png resize: (231, 281) 1356321768 -2.235820044912874 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329192_0.png resize: (151, 154) 1356321769 -3.898752456928391 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329193_0.png resize: (112, 132) 1356321770 -4.076762612540383 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329189_0.png resize: (215, 78) 1356321771 -1.3767867319054388 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329194_0.png resize: (510, 830) 1356321772 -2.9348953127858395 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329190_0.png resize: (211, 173) 1356321773 -3.4217437122458554 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329184_0.png resize: (323, 133) 1356321774 -4.100053551643327 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329216_0.png resize: (294, 51) 1356321775 -1.8306911340680587 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329210_0.png resize: (290, 528) 1356321776 -1.8389317120858317 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329212_0.png resize: (170, 98) 1356321777 -1.9723207321015563 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329203_0.png resize: (314, 435) 1356321778 -2.82339560534193 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329215_0.png resize: (191, 267) 1356321779 -1.3699396262839347 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329204_0.png resize: (235, 488) 1356321780 -1.480329234987543 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329211_0.png resize: (110, 141) 1356321781 -3.6036072274940008 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329218_0.png resize: (191, 78) 1356321782 0.19438976640150815 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329214_0.png resize: (189, 154) 1356321783 -1.680779050774973 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329213_0.png resize: (320, 130) 1356321784 -4.2043728069635415 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329232_0.png resize: (645, 544) 1356321785 -0.41755150786724915 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329237_0.png resize: (205, 70) 1356321786 0.02524540963958944 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329235_0.png resize: (168, 79) 1356321787 -1.845267832638661 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329225_0.png resize: (67, 100) 1356321788 -1.117487876009778 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329239_0.png resize: (108, 114) 1356321789 -1.0149494900795193 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329228_0.png resize: (418, 542) 1356321790 -0.11187980472295307 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329222_0.png resize: (357, 530) 1356321791 -0.05253543198768661 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329238_0.png resize: (412, 392) 1356321792 -0.4081483573153383 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329234_0.png resize: (159, 97) 1356321793 1.6583895341111021 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329220_0.png resize: (201, 247) 1356321794 0.6063622575773296 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329221_0.png resize: (200, 434) 1356321795 -1.0632692030848496 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329224_0.png resize: (236, 365) 1356321796 -0.4912394190438977 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329231_0.png resize: (236, 93) 1356321797 -0.10442015109082516 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329236_0.png resize: (164, 155) 1356321798 -0.8636002870953489 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329249_0.png resize: (302, 234) 1356321799 -2.1880213627491654 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329247_0.png resize: (434, 365) 1356321800 -1.19246604334873 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329243_0.png resize: (214, 327) 1356321801 -2.1456459432924797 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329244_0.png resize: (635, 707) 1356321802 -1.5892264120439263 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329241_0.png resize: (534, 669) 1356321803 -0.5278005688336252 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329242_0.png resize: (400, 212) 1356321804 -0.9674295574412426 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329248_0.png resize: (71, 82) 1356321805 -1.7647255057496782 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329255_0.png resize: (79, 175) 1356321806 -0.9929901825839764 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329252_0.png resize: (203, 148) 1356321807 -0.6387965171547171 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329253_0.png resize: (221, 184) 1356321808 -1.2607601993124136 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329257_0.png resize: (299, 275) 1356321809 -1.6888446668633263 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329246_0.png resize: (527, 493) 1356321810 -0.46155916408260556 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329258_0.png resize: (291, 146) 1356321811 2.3599728803786157 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329266_0.png resize: (183, 304) 1356321812 -2.914231194157561 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329260_0.png resize: (278, 418) 1356321813 -1.2793814166698871 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329264_0.png resize: (160, 105) 1356321814 -3.6787827635985884 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329259_0.png resize: (145, 324) 1356321815 -1.0428638241791952 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329263_0.png resize: (340, 170) 1356321816 -2.6439093171735792 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329272_0.png resize: (103, 197) 1356321817 -1.1283076475521567 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329261_0.png resize: (118, 162) 1356321818 -1.792289320644574 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329265_0.png resize: (259, 184) 1356321819 -2.4119499053373676 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329262_0.png resize: (106, 87) 1356321820 -0.4882372579925633 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329267_0.png resize: (206, 205) 1356321821 -2.3374721447366578 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329277_0.png resize: (262, 183) 1356321822 -1.572025264892755 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329275_0.png resize: (154, 225) 1356321823 -1.1980868420832123 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329293_0.png resize: (324, 450) 1356321824 -0.15746545451003288 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329284_0.png resize: (181, 167) 1356321825 1.0550688622714586 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329278_0.png resize: (131, 195) 1356321826 1.0826477776663281 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329282_0.png resize: (216, 197) 1356321827 -0.4574748311916588 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329290_0.png resize: (98, 163) 1356321829 -0.27195097464091744 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329285_0.png resize: (158, 155) 1356321830 -0.6265226590165874 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329291_0.png resize: (196, 184) 1356321831 -0.39632128037954545 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329287_0.png resize: (104, 106) 1356321832 -1.8424214082460122 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329295_0.png resize: (223, 196) 1356321833 -0.9529371348910155 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329296_0.png resize: (273, 244) 1356321834 -1.5230945356479118 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329300_0.png resize: (205, 293) 1356321835 -1.075119552208754 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329297_0.png resize: (174, 171) 1356321836 -0.8993093147563666 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329306_0.png resize: (639, 487) 1356321837 -0.4057593195881242 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329309_0.png resize: (157, 169) 1356321838 -0.9778767842341866 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329302_0.png resize: (333, 390) 1356321839 0.1811897979246135 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329304_0.png resize: (115, 200) 1356321840 -0.7430964671925556 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329305_0.png resize: (319, 168) 1356321841 -1.3048624114210499 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329308_0.png resize: (111, 231) 1356321842 1.5367545971741865 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329185_0.png resize: (204, 215) 1356321851 -1.6424446114046556 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329186_0.png resize: (717, 442) 1356321853 -1.6134873962848006 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329198_0.png resize: (174, 153) 1356321854 -1.760742823626587 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329208_0.png resize: (571, 811) 1356321855 -1.5503354077447546 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329201_0.png resize: (224, 297) 1356321856 -2.0284399190543003 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329209_0.png resize: (734, 474) 1356321857 -1.3181565308294814 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329223_0.png resize: (214, 241) 1356321858 -1.0433416080341333 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329240_0.png resize: (86, 127) 1356321859 1.2481097702461061 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329229_0.png resize: (747, 806) 1356321860 -1.2816627435524313 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329245_0.png resize: (141, 264) 1356321861 -1.40190548945857 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329256_0.png resize: (330, 290) 1356321862 -1.6012531675994492 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329251_0.png resize: (235, 263) 1356321863 -0.4117909793034821 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329254_0.png resize: (185, 164) 1356321864 -1.4494630559054045 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329269_0.png resize: (266, 159) 1356321865 -1.6424791325404677 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329268_0.png resize: (128, 204) 1356321866 -2.204834200494544 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329273_0.png resize: (165, 224) 1356321867 -0.8757535918478235 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329281_0.png resize: (129, 139) 1356321868 -0.45497928311546376 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329286_0.png resize: (146, 124) 1356321869 -0.4052594960867134 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329283_0.png resize: (163, 156) 1356321870 -0.08290929791022886 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329274_0.png resize: (333, 169) 1356321871 -0.9167620510620434 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329276_0.png resize: (195, 181) 1356321872 -1.1486740089045346 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329303_0.png resize: (229, 244) 1356321873 -0.39790140932613977 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329298_0.png resize: (180, 178) 1356321875 -3.709239658260054 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329196_0.png resize: (620, 471) 1356321936 -1.1428152591758185 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329183_0.png resize: (490, 901) 1356321938 -2.882490098882154 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329188_0.png resize: (323, 315) 1356321939 -2.574038045732677 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329197_0.png resize: (129, 143) 1356321941 -3.852678754435359 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329191_0.png resize: (441, 445) 1356321943 -3.0209253561298017 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329179_0.png resize: (399, 537) 1356321945 -4.331884659773115 treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329177_0.png resize: (468, 179) 1356321946 -4.382766213514781 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329219_0.png resize: (327, 419) 1356321947 -2.6375604203281706 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329207_0.png resize: (233, 272) 1356321949 -3.810502343016375 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329206_0.png resize: (394, 421) 1356321950 -2.5538504943980653 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329205_0.png resize: (194, 454) 1356321951 -2.6139868289797317 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329202_0.png resize: (427, 572) 1356321953 -4.029300231162201 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329200_0.png resize: (462, 173) 1356321954 -4.160545815482817 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329226_0.png resize: (222, 605) 1356321955 -1.5276400057415909 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329227_0.png resize: (270, 310) 1356321957 -1.9693797027595694 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329233_0.png resize: (385, 431) 1356321958 -0.9301356758759071 treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329230_0.png resize: (94, 221) 1356321959 -0.7643206725824604 treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329250_0.png resize: (316, 421) 1356321961 -2.715186719558575 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329270_0.png resize: (122, 112) 1356321962 -1.8576706283229614 treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329271_0.png resize: (446, 475) 1356321963 -2.7300849798471054 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329280_0.png resize: (251, 282) 1356321964 -2.4038877600421427 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329289_0.png resize: (276, 338) 1356321966 -2.5936067110586047 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329288_0.png resize: (111, 251) 1356321967 -1.0598167763363853 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329279_0.png resize: (337, 410) 1356321968 -1.775648991259594 treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329292_0.png resize: (320, 358) 1356321970 -1.048137473826541 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329294_0.png resize: (300, 405) 1356321971 -2.584106056997612 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329301_0.png resize: (365, 399) 1356321972 -2.0462749527924107 treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329299_0.png resize: (113, 175) 1356321974 -0.6485349237123006 treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329217_0.png resize: (266, 147) 1356321988 -3.4519538445303444 treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329307_0.png resize: (238, 419) 1356321989 -1.3734036827979021 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 141 time used for this insertion : 0.020509719848632812 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 141 time used for this insertion : 0.03672289848327637 save missing photos in datou_result : time spend for datou_step_exec : 30.63700819015503 time spend to save output : 0.06281208992004395 total time spend for step 6 : 30.699820280075073 step7:brightness Tue May 6 10:24:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111.jpg treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec.jpg treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d.jpg treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6.jpg treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63.jpg treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766.jpg treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998.jpg treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c.jpg treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329180_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329181_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329178_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329182_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329195_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329187_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329199_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329192_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329193_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329189_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329194_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329190_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329184_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329216_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329210_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329212_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329203_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329215_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329204_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329211_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329218_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329214_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329213_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329232_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329237_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329235_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329225_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329239_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329228_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329222_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329238_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329234_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329220_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329221_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329224_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329231_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329236_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329249_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329247_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329243_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329244_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329241_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329242_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329248_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329255_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329252_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329253_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329257_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329246_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329258_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329266_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329260_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329264_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329259_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329263_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329272_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329261_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329265_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329262_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329267_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329277_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329275_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329293_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329284_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329278_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329282_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329290_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329285_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329291_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329287_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329295_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329296_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329300_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329297_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329306_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329309_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329302_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329304_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329305_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329308_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329185_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329186_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329198_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329208_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329201_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329209_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329223_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329240_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329229_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329245_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329256_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329251_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329254_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329269_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329268_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329273_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329281_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329286_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329283_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329274_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329276_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329303_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329298_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329196_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329183_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329188_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329197_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329191_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329179_0.png treat image : temp/1746519629_317658_1356308633_80e48334b544054916454de7bf43e111_rle_crop_3786329177_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329219_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329207_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329206_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329205_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329202_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329200_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329226_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329227_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329233_0.png treat image : temp/1746519629_317658_1356308628_2fcc095fba29d8f4da8ce90ded8d658d_rle_crop_3786329230_0.png treat image : temp/1746519629_317658_1356308580_72795abfe2ad88eb99c60a9d8ca7bde6_rle_crop_3786329250_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329270_0.png treat image : temp/1746519629_317658_1356308576_56a552ebe6d0868086dbae64318b3e63_rle_crop_3786329271_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329280_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329289_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329288_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329279_0.png treat image : temp/1746519629_317658_1356308571_8b37f5577efb096efe8034fd1cda6766_rle_crop_3786329292_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329294_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329301_0.png treat image : temp/1746519629_317658_1356308566_aa2f926bd5e936445be6574e8dcfa998_rle_crop_3786329299_0.png treat image : temp/1746519629_317658_1356308630_5cce8e77774d20bd26af0bee93d484ec_rle_crop_3786329217_0.png treat image : temp/1746519629_317658_1356308531_12d612f723b705aaf9129567abdd0b5c_rle_crop_3786329307_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 141 time used for this insertion : 0.02246999740600586 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 141 time used for this insertion : 0.03287863731384277 save missing photos in datou_result : time spend for datou_step_exec : 8.153119802474976 time spend to save output : 0.06147360801696777 total time spend for step 7 : 8.214593410491943 step8:velours_tree Tue May 6 10:25:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.167339563369751 time spend to save output : 4.7206878662109375e-05 total time spend for step 8 : 1.167386770248413 step9:send_mail_cod Tue May 6 10:25:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22712582_06-05-2025_10_25_09.pdf 22712621 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227126211746519909 22712622 imagette227126221746519910 22712623 imagette227126231746519910 22712624 imagette227126241746519910 22712625 change filename to text .change filename to text .imagette227126251746519910 22712626 imagette227126261746519910 22712627 change filename to text .imagette227126271746519910 22712629 imagette227126291746519911 22712630 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227126301746519911 22712631 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227126311746519912 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22712582 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22712621,22712622,22712623,22712624,22712625,22712626,22712627,22712628,22712629,22712630,22712631?tags=pet_clair,autre,background,pet_fonce,pehd,mal_croppe,metal,environnement,flou,papier,carton args[1356308633] : ((1356308633, -5.452549631917293, 492609224), (1356308633, 0.26585878624747267, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308630] : ((1356308630, -5.342885029736863, 492609224), (1356308630, 0.232768461419369, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308628] : ((1356308628, -1.087307808850906, 492688767), (1356308628, 0.3874254904359795, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308580] : ((1356308580, -2.7073552958617118, 492609224), (1356308580, -0.026045692833453805, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308576] : ((1356308576, -3.8117594391348697, 492609224), (1356308576, 0.19365602190734343, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308571] : ((1356308571, -2.121207087631017, 492609224), (1356308571, 0.2560436498975047, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308566] : ((1356308566, -1.955837135530683, 492688767), (1356308566, -0.09337181737809165, 496442774), '0.1363949844260621') We are sending mail with results at report@fotonower.com args[1356308531] : ((1356308531, 1.480653038756646, 492688767), (1356308531, 0.151351249051493, 2107752395), '0.1363949844260621') We are sending mail with results at report@fotonower.com refus_total : 0.1363949844260621 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=22712582 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22712582_06-05-2025_10_25_09.pdf results_Auto_P22712582_06-05-2025_10_25_09.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22712582_06-05-2025_10_25_09.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','22712582','results_Auto_P22712582_06-05-2025_10_25_09.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22712582_06-05-2025_10_25_09.pdf','pdf','','0.67','0.1363949844260621') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/22712582

https://www.fotonower.com/image?json=false&list_photos_id=1356308633
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308630
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308628
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308580
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308576
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308571
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308566
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1356308531
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.480653038756646)

Dans ces conditions,le taux de refus est: 13.64%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/22712621?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/22712625?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/22712627?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/22712630?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/22712631?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22712582_06-05-2025_10_25_09.pdf.

Lien vers velours :https://www.fotonower.com/velours/22712621,22712622,22712623,22712624,22712625,22712626,22712627,22712628,22712629,22712630,22712631?tags=pet_clair,autre,background,pet_fonce,pehd,mal_croppe,metal,environnement,flou,papier,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 06 May 2025 08:25:16 GMT Content-Length: 0 Connection: close X-Message-Id: 3GgReiq9SJGdnkNmGhgCNg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356308633, 1356308630, 1356308628, 1356308580, 1356308576, 1356308571, 1356308566, 1356308531] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308633', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308630', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308628', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308580', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308576', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308571', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308566', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308531', None, None, None, None, None, '2797056') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.01362156867980957 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.835390567779541 time spend to save output : 0.013893842697143555 total time spend for step 9 : 6.849284410476685 step10:split_time_score Tue May 6 10:25:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 06052025 22712582 Nombre de photos uploadées : 8 / 23040 (0%) 06052025 22712582 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 06052025 22712582 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0005321502685546875 elapsed_time : insert_dashboard_record_day_entry 0.02353692054748535 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.10237958578664899 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22709064_06-05-2025_08_12_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22709064 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22709064 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22712580 order by id desc limit 1 Qualite : 0.1363949844260621 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22712582_06-05-2025_10_25_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22712582 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22712582 AND mptpi.`type`=3594 To do Qualite : 0.021461336783706136 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22711961_06-05-2025_09_43_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22711961 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22711961 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'06052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356308633, 1356308630, 1356308628, 1356308580, 1356308576, 1356308571, 1356308566, 1356308531] Looping around the photos to save general results len do output : 1 /22712582Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308633', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308630', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308628', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308580', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308576', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308571', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308566', None, None, None, None, None, '2797056') ('3318', None, None, None, None, None, None, None, '2797056') ('3318', '22712582', '1356308531', None, None, None, None, None, '2797056') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.019336223602294922 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.243786334991455 time spend to save output : 0.019488096237182617 total time spend for step 10 : 1.2632744312286377 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 129.90user 116.36system 4:52.57elapsed 84%CPU (0avgtext+0avgdata 4901248maxresident)k 915944inputs+91056outputs (18578major+11947683minor)pagefaults 0swaps