python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3297538 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2984097'] with mtr_portfolio_ids : ['23441275'] and first list_photo_ids : [] new path : /proc/3297538/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 1.3256497383117676 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu May 29 17:50:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-29 17:50:34.002066: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-29 17:50:34.035306: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-29 17:50:34.037812: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7e78000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-29 17:50:34.037893: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-29 17:50:34.043297: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-29 17:50:34.285534: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x6100d20 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-29 17:50:34.285602: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-29 17:50:34.287134: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-29 17:50:34.288023: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-29 17:50:34.293604: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-29 17:50:34.296383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-29 17:50:34.296798: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-29 17:50:34.300914: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-29 17:50:34.302332: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-29 17:50:34.309357: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-29 17:50:34.310883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-29 17:50:34.311003: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-29 17:50:34.311833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-29 17:50:34.311849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-29 17:50:34.311859: I tensorflow/core/common_ruInside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-29 17:50:47.694357: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-29 17:50:47.878085: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 40.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 23.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 Detection mask done ! Trying to reset tf kernel 3298082 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5485 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10774 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0031800270080566406 nb_pixel_total : 38682 time to create 1 rle with old method : 0.046813011169433594 length of segment : 317 time for calcul the mask position with numpy : 0.0039052963256835938 nb_pixel_total : 84856 time to create 1 rle with old method : 0.1658644676208496 length of segment : 540 time for calcul the mask position with numpy : 0.0013842582702636719 nb_pixel_total : 38033 time to create 1 rle with old method : 0.04356503486633301 length of segment : 166 time for calcul the mask position with numpy : 0.0011644363403320312 nb_pixel_total : 34471 time to create 1 rle with old method : 0.04868888854980469 length of segment : 251 time for calcul the mask position with numpy : 0.0021674633026123047 nb_pixel_total : 71632 time to create 1 rle with old method : 0.09080743789672852 length of segment : 252 time for calcul the mask position with numpy : 0.0018603801727294922 nb_pixel_total : 59505 time to create 1 rle with old method : 0.10967350006103516 length of segment : 380 time for calcul the mask position with numpy : 0.0007259845733642578 nb_pixel_total : 15677 time to create 1 rle with old method : 0.026795625686645508 length of segment : 125 time for calcul the mask position with numpy : 0.0216214656829834 nb_pixel_total : 316347 time to create 1 rle with new method : 0.223891019821167 length of segment : 1458 time for calcul the mask position with numpy : 0.01084589958190918 nb_pixel_total : 326214 time to create 1 rle with new method : 0.024195194244384766 length of segment : 848 time for calcul the mask position with numpy : 0.0028743743896484375 nb_pixel_total : 75173 time to create 1 rle with old method : 0.08815670013427734 length of segment : 393 time for calcul the mask position with numpy : 0.0029973983764648438 nb_pixel_total : 77752 time to create 1 rle with old method : 0.0895085334777832 length of segment : 352 time for calcul the mask position with numpy : 0.0012509822845458984 nb_pixel_total : 73258 time to create 1 rle with old method : 0.08936452865600586 length of segment : 337 time for calcul the mask position with numpy : 0.0018885135650634766 nb_pixel_total : 57343 time to create 1 rle with old method : 0.0703885555267334 length of segment : 499 time for calcul the mask position with numpy : 0.033643484115600586 nb_pixel_total : 389069 time to create 1 rle with new method : 0.04485440254211426 length of segment : 933 time for calcul the mask position with numpy : 0.01608872413635254 nb_pixel_total : 403587 time to create 1 rle with new method : 0.0702812671661377 length of segment : 1320 time for calcul the mask position with numpy : 0.0008792877197265625 nb_pixel_total : 33869 time to create 1 rle with old method : 0.037934064865112305 length of segment : 301 time for calcul the mask position with numpy : 0.0009381771087646484 nb_pixel_total : 55620 time to create 1 rle with old method : 0.06698060035705566 length of segment : 188 time for calcul the mask position with numpy : 0.0024454593658447266 nb_pixel_total : 69708 time to create 1 rle with old method : 0.08359003067016602 length of segment : 273 time for calcul the mask position with numpy : 0.0007445812225341797 nb_pixel_total : 32943 time to create 1 rle with old method : 0.03735852241516113 length of segment : 283 time for calcul the mask position with numpy : 0.0002815723419189453 nb_pixel_total : 12436 time to create 1 rle with old method : 0.020635128021240234 length of segment : 105 time for calcul the mask position with numpy : 0.0018298625946044922 nb_pixel_total : 84564 time to create 1 rle with old method : 0.12525177001953125 length of segment : 331 time for calcul the mask position with numpy : 0.00041866302490234375 nb_pixel_total : 22021 time to create 1 rle with old method : 0.0252835750579834 length of segment : 156 time for calcul the mask position with numpy : 0.0003445148468017578 nb_pixel_total : 4019 time to create 1 rle with old method : 0.005026102066040039 length of segment : 256 time for calcul the mask position with numpy : 0.0028181076049804688 nb_pixel_total : 62626 time to create 1 rle with old method : 0.07100319862365723 length of segment : 344 time for calcul the mask position with numpy : 0.0026738643646240234 nb_pixel_total : 76723 time to create 1 rle with old method : 0.11177968978881836 length of segment : 401 time for calcul the mask position with numpy : 0.0008378028869628906 nb_pixel_total : 11226 time to create 1 rle with old method : 0.014070749282836914 length of segment : 215 time for calcul the mask position with numpy : 0.0012090206146240234 nb_pixel_total : 31432 time to create 1 rle with old method : 0.03883028030395508 length of segment : 214 time for calcul the mask position with numpy : 0.002408266067504883 nb_pixel_total : 69541 time to create 1 rle with old method : 0.07896757125854492 length of segment : 405 time for calcul the mask position with numpy : 0.0012693405151367188 nb_pixel_total : 30892 time to create 1 rle with old method : 0.03523659706115723 length of segment : 201 time for calcul the mask position with numpy : 0.02593374252319336 nb_pixel_total : 553891 time to create 1 rle with new method : 0.09113693237304688 length of segment : 1477 time for calcul the mask position with numpy : 0.004258871078491211 nb_pixel_total : 154042 time to create 1 rle with new method : 0.005554676055908203 length of segment : 421 time for calcul the mask position with numpy : 0.0014379024505615234 nb_pixel_total : 58067 time to create 1 rle with old method : 0.07105827331542969 length of segment : 265 time for calcul the mask position with numpy : 0.0005624294281005859 nb_pixel_total : 15013 time to create 1 rle with old method : 0.017507076263427734 length of segment : 167 time for calcul the mask position with numpy : 0.005367279052734375 nb_pixel_total : 187102 time to create 1 rle with new method : 0.010239362716674805 length of segment : 728 time for calcul the mask position with numpy : 0.007819652557373047 nb_pixel_total : 92619 time to create 1 rle with old method : 0.1078341007232666 length of segment : 422 time for calcul the mask position with numpy : 0.0005650520324707031 nb_pixel_total : 13886 time to create 1 rle with old method : 0.01623702049255371 length of segment : 144 time for calcul the mask position with numpy : 0.0026230812072753906 nb_pixel_total : 161773 time to create 1 rle with new method : 0.007536411285400391 length of segment : 529 time for calcul the mask position with numpy : 0.0006113052368164062 nb_pixel_total : 22515 time to create 1 rle with old method : 0.026659250259399414 length of segment : 117 time for calcul the mask position with numpy : 0.0005812644958496094 nb_pixel_total : 13525 time to create 1 rle with old method : 0.015613794326782227 length of segment : 205 time for calcul the mask position with numpy : 0.001802682876586914 nb_pixel_total : 47333 time to create 1 rle with old method : 0.055069923400878906 length of segment : 263 time for calcul the mask position with numpy : 0.018678665161132812 nb_pixel_total : 648770 time to create 1 rle with new method : 0.09386730194091797 length of segment : 1081 time for calcul the mask position with numpy : 0.00179290771484375 nb_pixel_total : 35965 time to create 1 rle with old method : 0.04094338417053223 length of segment : 383 time for calcul the mask position with numpy : 0.0007529258728027344 nb_pixel_total : 35122 time to create 1 rle with old method : 0.04029345512390137 length of segment : 203 time for calcul the mask position with numpy : 0.0024521350860595703 nb_pixel_total : 46635 time to create 1 rle with old method : 0.05271315574645996 length of segment : 417 time for calcul the mask position with numpy : 0.0007717609405517578 nb_pixel_total : 18208 time to create 1 rle with old method : 0.020872831344604492 length of segment : 250 time for calcul the mask position with numpy : 0.0005700588226318359 nb_pixel_total : 13014 time to create 1 rle with old method : 0.015174627304077148 length of segment : 167 time for calcul the mask position with numpy : 0.00043702125549316406 nb_pixel_total : 11295 time to create 1 rle with old method : 0.013482093811035156 length of segment : 85 time for calcul the mask position with numpy : 0.0012736320495605469 nb_pixel_total : 37542 time to create 1 rle with old method : 0.04251980781555176 length of segment : 267 time for calcul the mask position with numpy : 0.0003666877746582031 nb_pixel_total : 12529 time to create 1 rle with old method : 0.016225576400756836 length of segment : 115 time for calcul the mask position with numpy : 0.0002713203430175781 nb_pixel_total : 5778 time to create 1 rle with old method : 0.006807088851928711 length of segment : 119 time for calcul the mask position with numpy : 0.0006353855133056641 nb_pixel_total : 34047 time to create 1 rle with old method : 0.03928875923156738 length of segment : 233 time for calcul the mask position with numpy : 0.012561798095703125 nb_pixel_total : 472090 time to create 1 rle with new method : 0.04365801811218262 length of segment : 976 time for calcul the mask position with numpy : 0.00106048583984375 nb_pixel_total : 38281 time to create 1 rle with old method : 0.04638171195983887 length of segment : 270 time for calcul the mask position with numpy : 0.0025107860565185547 nb_pixel_total : 101257 time to create 1 rle with old method : 0.11317062377929688 length of segment : 251 time for calcul the mask position with numpy : 0.0011777877807617188 nb_pixel_total : 41334 time to create 1 rle with old method : 0.04811286926269531 length of segment : 257 time for calcul the mask position with numpy : 0.0012650489807128906 nb_pixel_total : 38075 time to create 1 rle with old method : 0.04399466514587402 length of segment : 203 time for calcul the mask position with numpy : 0.0004677772521972656 nb_pixel_total : 10832 time to create 1 rle with old method : 0.012809514999389648 length of segment : 89 time for calcul the mask position with numpy : 0.0010135173797607422 nb_pixel_total : 30927 time to create 1 rle with old method : 0.037293434143066406 length of segment : 270 time for calcul the mask position with numpy : 0.0017447471618652344 nb_pixel_total : 41048 time to create 1 rle with old method : 0.04630398750305176 length of segment : 402 time for calcul the mask position with numpy : 0.0009660720825195312 nb_pixel_total : 17118 time to create 1 rle with old method : 0.019814014434814453 length of segment : 234 time for calcul the mask position with numpy : 0.0006155967712402344 nb_pixel_total : 17567 time to create 1 rle with old method : 0.029123306274414062 length of segment : 224 time for calcul the mask position with numpy : 0.003040790557861328 nb_pixel_total : 80527 time to create 1 rle with old method : 0.14598321914672852 length of segment : 351 time for calcul the mask position with numpy : 0.0011644363403320312 nb_pixel_total : 26761 time to create 1 rle with old method : 0.03196597099304199 length of segment : 179 time for calcul the mask position with numpy : 0.0022885799407958984 nb_pixel_total : 73359 time to create 1 rle with old method : 0.08329391479492188 length of segment : 343 time for calcul the mask position with numpy : 0.0005657672882080078 nb_pixel_total : 14093 time to create 1 rle with old method : 0.01641559600830078 length of segment : 171 time for calcul the mask position with numpy : 0.0005230903625488281 nb_pixel_total : 15224 time to create 1 rle with old method : 0.01816415786743164 length of segment : 89 time for calcul the mask position with numpy : 0.0012309551239013672 nb_pixel_total : 30439 time to create 1 rle with old method : 0.03488636016845703 length of segment : 228 time for calcul the mask position with numpy : 0.0005543231964111328 nb_pixel_total : 11035 time to create 1 rle with old method : 0.012749910354614258 length of segment : 164 time for calcul the mask position with numpy : 0.0006954669952392578 nb_pixel_total : 16659 time to create 1 rle with old method : 0.020154953002929688 length of segment : 152 time for calcul the mask position with numpy : 0.002196073532104492 nb_pixel_total : 67195 time to create 1 rle with old method : 0.07588648796081543 length of segment : 312 time for calcul the mask position with numpy : 0.001556396484375 nb_pixel_total : 47284 time to create 1 rle with old method : 0.07220745086669922 length of segment : 292 time for calcul the mask position with numpy : 0.0006945133209228516 nb_pixel_total : 18840 time to create 1 rle with old method : 0.021756649017333984 length of segment : 187 time for calcul the mask position with numpy : 0.0014154911041259766 nb_pixel_total : 36073 time to create 1 rle with old method : 0.04157590866088867 length of segment : 361 time for calcul the mask position with numpy : 0.003710508346557617 nb_pixel_total : 114047 time to create 1 rle with old method : 0.1381540298461914 length of segment : 504 time for calcul the mask position with numpy : 0.0005185604095458984 nb_pixel_total : 12227 time to create 1 rle with old method : 0.014603376388549805 length of segment : 89 time for calcul the mask position with numpy : 0.0024480819702148438 nb_pixel_total : 72481 time to create 1 rle with old method : 0.08299493789672852 length of segment : 275 time for calcul the mask position with numpy : 0.0009429454803466797 nb_pixel_total : 26671 time to create 1 rle with old method : 0.030041933059692383 length of segment : 207 time for calcul the mask position with numpy : 0.001917123794555664 nb_pixel_total : 58445 time to create 1 rle with old method : 0.06656718254089355 length of segment : 274 time for calcul the mask position with numpy : 0.0027968883514404297 nb_pixel_total : 66669 time to create 1 rle with old method : 0.07462525367736816 length of segment : 348 time for calcul the mask position with numpy : 0.001138448715209961 nb_pixel_total : 28732 time to create 1 rle with old method : 0.03378486633300781 length of segment : 230 time for calcul the mask position with numpy : 0.0013082027435302734 nb_pixel_total : 38002 time to create 1 rle with old method : 0.04353189468383789 length of segment : 220 time for calcul the mask position with numpy : 0.0013298988342285156 nb_pixel_total : 28640 time to create 1 rle with old method : 0.03391599655151367 length of segment : 171 time for calcul the mask position with numpy : 0.0011296272277832031 nb_pixel_total : 39366 time to create 1 rle with old method : 0.04525589942932129 length of segment : 213 time for calcul the mask position with numpy : 0.0021409988403320312 nb_pixel_total : 40976 time to create 1 rle with old method : 0.04660320281982422 length of segment : 385 time for calcul the mask position with numpy : 0.0004584789276123047 nb_pixel_total : 9520 time to create 1 rle with old method : 0.011030197143554688 length of segment : 104 time for calcul the mask position with numpy : 0.0029664039611816406 nb_pixel_total : 62191 time to create 1 rle with old method : 0.06932187080383301 length of segment : 468 time for calcul the mask position with numpy : 0.0005083084106445312 nb_pixel_total : 15383 time to create 1 rle with old method : 0.018184661865234375 length of segment : 112 time for calcul the mask position with numpy : 0.002169370651245117 nb_pixel_total : 74261 time to create 1 rle with old method : 0.08407282829284668 length of segment : 508 time for calcul the mask position with numpy : 0.0027055740356445312 nb_pixel_total : 74125 time to create 1 rle with old method : 0.08358287811279297 length of segment : 737 time for calcul the mask position with numpy : 0.0019614696502685547 nb_pixel_total : 60302 time to create 1 rle with old method : 0.07247376441955566 length of segment : 238 time for calcul the mask position with numpy : 0.001592874526977539 nb_pixel_total : 41937 time to create 1 rle with old method : 0.04700922966003418 length of segment : 315 time for calcul the mask position with numpy : 0.0009720325469970703 nb_pixel_total : 29078 time to create 1 rle with old method : 0.03566241264343262 length of segment : 261 time for calcul the mask position with numpy : 0.002779245376586914 nb_pixel_total : 107433 time to create 1 rle with old method : 0.11996674537658691 length of segment : 333 time for calcul the mask position with numpy : 0.00034689903259277344 nb_pixel_total : 8815 time to create 1 rle with old method : 0.01049351692199707 length of segment : 121 time for calcul the mask position with numpy : 0.0005640983581542969 nb_pixel_total : 20769 time to create 1 rle with old method : 0.02738189697265625 length of segment : 108 time for calcul the mask position with numpy : 0.0050160884857177734 nb_pixel_total : 162913 time to create 1 rle with new method : 0.00953221321105957 length of segment : 293 time for calcul the mask position with numpy : 0.0005633831024169922 nb_pixel_total : 16532 time to create 1 rle with old method : 0.0192873477935791 length of segment : 120 time for calcul the mask position with numpy : 0.0003829002380371094 nb_pixel_total : 5689 time to create 1 rle with old method : 0.008450746536254883 length of segment : 88 time spent for convertir_results : 11.440913915634155 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 98 chid ids of type : 3594 Number RLEs to save : 32629 save missing photos in datou_result : time spend for datou_step_exec : 66.37262916564941 time spend to save output : 1.825007438659668 total time spend for step 1 : 68.19763660430908 step2:crop_condition Thu May 29 17:51:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 98 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 45 About to insert : list_path_to_insert length 45 new photo from crops ! About to upload 45 photos upload in portfolio : 3736932 init cache_photo without model_param we have 45 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748533916_3297538 we have uploaded 45 photos in the portfolio 3736932 time of upload the photos Elapsed time : 10.518731355667114 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! About to upload 18 photos upload in portfolio : 3736932 init cache_photo without model_param we have 18 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748533935_3297538 we have uploaded 18 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.14171290397644 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 3736932 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748533952_3297538 we have uploaded 32 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.435534954071045 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748533962_3297538 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.431044101715088 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1361633751, 1361633240, 1361633234, 1361633174, 1361633171, 1361633166, 1361633164, 1361633161, 1361633155] Looping around the photos to save general results len do output : 98 /1361650044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361650180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633751', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633240', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633234', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633174', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633171', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633166', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633164', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633161', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633155', None, None, None, None, None, '2984097') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 303 time used for this insertion : 0.0253908634185791 save_final save missing photos in datou_result : time spend for datou_step_exec : 64.88268685340881 time spend to save output : 0.028296232223510742 total time spend for step 2 : 64.91098308563232 step3:rle_unique_nms_with_priority Thu May 29 17:52:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 98 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 5.130286693572998 time for calcul the mask position with numpy : 0.7264299392700195 nb_pixel_total : 5837434 time to create 1 rle with new method : 0.5723989009857178 time for calcul the mask position with numpy : 0.03559517860412598 nb_pixel_total : 77221 time to create 1 rle with old method : 0.0864870548248291 time for calcul the mask position with numpy : 0.03497600555419922 nb_pixel_total : 13938 time to create 1 rle with old method : 0.01737189292907715 time for calcul the mask position with numpy : 0.03190946578979492 nb_pixel_total : 11313 time to create 1 rle with old method : 0.012991189956665039 time for calcul the mask position with numpy : 0.03283572196960449 nb_pixel_total : 77752 time to create 1 rle with old method : 0.09170770645141602 time for calcul the mask position with numpy : 0.03150153160095215 nb_pixel_total : 75173 time to create 1 rle with old method : 0.08307099342346191 time for calcul the mask position with numpy : 0.024449586868286133 nb_pixel_total : 326214 time to create 1 rle with new method : 0.6947848796844482 time for calcul the mask position with numpy : 0.023283958435058594 nb_pixel_total : 288339 time to create 1 rle with new method : 0.5557050704956055 time for calcul the mask position with numpy : 0.02146148681640625 nb_pixel_total : 15677 time to create 1 rle with old method : 0.01749277114868164 time for calcul the mask position with numpy : 0.02121424674987793 nb_pixel_total : 59505 time to create 1 rle with old method : 0.06673312187194824 time for calcul the mask position with numpy : 0.02184915542602539 nb_pixel_total : 71632 time to create 1 rle with old method : 0.07852554321289062 time for calcul the mask position with numpy : 0.02198052406311035 nb_pixel_total : 34471 time to create 1 rle with old method : 0.04066348075866699 time for calcul the mask position with numpy : 0.025546789169311523 nb_pixel_total : 38033 time to create 1 rle with old method : 0.04280996322631836 time for calcul the mask position with numpy : 0.022832155227661133 nb_pixel_total : 84856 time to create 1 rle with old method : 0.10570144653320312 time for calcul the mask position with numpy : 0.02492237091064453 nb_pixel_total : 38682 time to create 1 rle with old method : 0.051650047302246094 create new chi : 3.697324514389038 time to delete rle : 0.030716657638549805 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 13875 TO DO : save crop sub photo not yet done ! save time : 0.8550140857696533 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 6.213647842407227 time for calcul the mask position with numpy : 0.4012293815612793 nb_pixel_total : 5483393 time to create 1 rle with new method : 0.42932939529418945 time for calcul the mask position with numpy : 0.02445673942565918 nb_pixel_total : 154042 time to create 1 rle with new method : 0.7313022613525391 time for calcul the mask position with numpy : 0.032746076583862305 nb_pixel_total : 553891 time to create 1 rle with new method : 0.6519677639007568 time for calcul the mask position with numpy : 0.021222829818725586 nb_pixel_total : 30892 time to create 1 rle with old method : 0.03406858444213867 time for calcul the mask position with numpy : 0.02111506462097168 nb_pixel_total : 12661 time to create 1 rle with old method : 0.014510869979858398 time for calcul the mask position with numpy : 0.021810531616210938 nb_pixel_total : 31432 time to create 1 rle with old method : 0.03490185737609863 time for calcul the mask position with numpy : 0.021953821182250977 nb_pixel_total : 7131 time to create 1 rle with old method : 0.007786750793457031 time for calcul the mask position with numpy : 0.02305912971496582 nb_pixel_total : 76723 time to create 1 rle with old method : 0.08300065994262695 time for calcul the mask position with numpy : 0.021730422973632812 nb_pixel_total : 62626 time to create 1 rle with old method : 0.06782245635986328 time for calcul the mask position with numpy : 0.021889448165893555 nb_pixel_total : 4019 time to create 1 rle with old method : 0.0045244693756103516 time for calcul the mask position with numpy : 0.020568370819091797 nb_pixel_total : 22021 time to create 1 rle with old method : 0.024031639099121094 time for calcul the mask position with numpy : 0.02165532112121582 nb_pixel_total : 3246 time to create 1 rle with old method : 0.0037560462951660156 time for calcul the mask position with numpy : 0.02074742317199707 nb_pixel_total : 12436 time to create 1 rle with old method : 0.013839960098266602 time for calcul the mask position with numpy : 0.021022558212280273 nb_pixel_total : 32943 time to create 1 rle with old method : 0.03553628921508789 time for calcul the mask position with numpy : 0.021760225296020508 nb_pixel_total : 69708 time to create 1 rle with old method : 0.08066773414611816 time for calcul the mask position with numpy : 0.02183222770690918 nb_pixel_total : 55620 time to create 1 rle with old method : 0.06470465660095215 time for calcul the mask position with numpy : 0.0357358455657959 nb_pixel_total : 33869 time to create 1 rle with old method : 0.03697657585144043 time for calcul the mask position with numpy : 0.038591861724853516 nb_pixel_total : 403587 time to create 1 rle with new method : 0.765397310256958 create new chi : 4.006918668746948 time to delete rle : 0.0022020339965820312 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 14852 TO DO : save crop sub photo not yet done ! save time : 0.9187872409820557 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 3.8499763011932373 time for calcul the mask position with numpy : 0.6849737167358398 nb_pixel_total : 6499265 time to create 1 rle with new method : 0.6717863082885742 time for calcul the mask position with numpy : 0.020975589752197266 nb_pixel_total : 22515 time to create 1 rle with old method : 0.02486395835876465 time for calcul the mask position with numpy : 0.02238917350769043 nb_pixel_total : 161773 time to create 1 rle with new method : 0.5830953121185303 time for calcul the mask position with numpy : 0.020676612854003906 nb_pixel_total : 13886 time to create 1 rle with old method : 0.015377044677734375 time for calcul the mask position with numpy : 0.03364825248718262 nb_pixel_total : 92619 time to create 1 rle with old method : 0.10162615776062012 time for calcul the mask position with numpy : 0.03320884704589844 nb_pixel_total : 187102 time to create 1 rle with new method : 0.4954073429107666 time for calcul the mask position with numpy : 0.024477720260620117 nb_pixel_total : 15013 time to create 1 rle with old method : 0.018220186233520508 time for calcul the mask position with numpy : 0.022164106369018555 nb_pixel_total : 58067 time to create 1 rle with old method : 0.06377625465393066 create new chi : 2.924549102783203 time to delete rle : 0.0009968280792236328 batch 1 Loaded 15 chid ids of type : 3594 +++++++++Number RLEs to save : 6904 TO DO : save crop sub photo not yet done ! save time : 0.42299962043762207 nb_obj : 13 nb_hashtags : 3 time to prepare the origin masks : 4.900092124938965 time for calcul the mask position with numpy : 0.6814699172973633 nb_pixel_total : 6159489 time to create 1 rle with new method : 0.7392590045928955 time for calcul the mask position with numpy : 0.022680044174194336 nb_pixel_total : 1793 time to create 1 rle with old method : 0.0027375221252441406 time for calcul the mask position with numpy : 0.02097296714782715 nb_pixel_total : 5778 time to create 1 rle with old method : 0.006410360336303711 time for calcul the mask position with numpy : 0.02221965789794922 nb_pixel_total : 12529 time to create 1 rle with old method : 0.02084493637084961 time for calcul the mask position with numpy : 0.0221402645111084 nb_pixel_total : 37542 time to create 1 rle with old method : 0.04205322265625 time for calcul the mask position with numpy : 0.02081775665283203 nb_pixel_total : 11295 time to create 1 rle with old method : 0.012325286865234375 time for calcul the mask position with numpy : 0.02107691764831543 nb_pixel_total : 13014 time to create 1 rle with old method : 0.01439809799194336 time for calcul the mask position with numpy : 0.020956039428710938 nb_pixel_total : 18208 time to create 1 rle with old method : 0.020199298858642578 time for calcul the mask position with numpy : 0.021571636199951172 nb_pixel_total : 46635 time to create 1 rle with old method : 0.05115222930908203 time for calcul the mask position with numpy : 0.021500587463378906 nb_pixel_total : 1201 time to create 1 rle with old method : 0.0013468265533447266 time for calcul the mask position with numpy : 0.021393537521362305 nb_pixel_total : 35965 time to create 1 rle with old method : 0.03950047492980957 time for calcul the mask position with numpy : 0.027579545974731445 nb_pixel_total : 645933 time to create 1 rle with new method : 0.7127079963684082 time for calcul the mask position with numpy : 0.03474712371826172 nb_pixel_total : 47333 time to create 1 rle with old method : 0.07115912437438965 time for calcul the mask position with numpy : 0.03687930107116699 nb_pixel_total : 13525 time to create 1 rle with old method : 0.015158653259277344 create new chi : 2.8196053504943848 time to delete rle : 0.0015337467193603516 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 8968 TO DO : save crop sub photo not yet done ! save time : 0.5506174564361572 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 2.723823070526123 time for calcul the mask position with numpy : 0.732358455657959 nb_pixel_total : 6348371 time to create 1 rle with new method : 0.861015796661377 time for calcul the mask position with numpy : 0.025789737701416016 nb_pixel_total : 10832 time to create 1 rle with old method : 0.012764930725097656 time for calcul the mask position with numpy : 0.02245354652404785 nb_pixel_total : 38075 time to create 1 rle with old method : 0.043103694915771484 time for calcul the mask position with numpy : 0.04033470153808594 nb_pixel_total : 41334 time to create 1 rle with old method : 0.10401391983032227 time for calcul the mask position with numpy : 0.05654120445251465 nb_pixel_total : 101257 time to create 1 rle with old method : 0.13607120513916016 time for calcul the mask position with numpy : 0.036005258560180664 nb_pixel_total : 38281 time to create 1 rle with old method : 0.05505514144897461 time for calcul the mask position with numpy : 0.04273533821105957 nb_pixel_total : 472090 time to create 1 rle with new method : 0.5567929744720459 create new chi : 2.794450283050537 time to delete rle : 0.0011515617370605469 batch 1 Loaded 13 chid ids of type : 3594 +++++++Number RLEs to save : 6252 TO DO : save crop sub photo not yet done ! save time : 0.4112131595611572 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 3.8624155521392822 time for calcul the mask position with numpy : 0.3603782653808594 nb_pixel_total : 6733616 time to create 1 rle with new method : 0.4174380302429199 time for calcul the mask position with numpy : 0.028063535690307617 nb_pixel_total : 15224 time to create 1 rle with old method : 0.01694035530090332 time for calcul the mask position with numpy : 0.020923376083374023 nb_pixel_total : 14093 time to create 1 rle with old method : 0.015692949295043945 time for calcul the mask position with numpy : 0.023371458053588867 nb_pixel_total : 73359 time to create 1 rle with old method : 0.08208179473876953 time for calcul the mask position with numpy : 0.021984577178955078 nb_pixel_total : 26761 time to create 1 rle with old method : 0.032172203063964844 time for calcul the mask position with numpy : 0.021398544311523438 nb_pixel_total : 80527 time to create 1 rle with old method : 0.09027481079101562 time for calcul the mask position with numpy : 0.020726919174194336 nb_pixel_total : 17567 time to create 1 rle with old method : 0.019439697265625 time for calcul the mask position with numpy : 0.021844863891601562 nb_pixel_total : 17118 time to create 1 rle with old method : 0.01922464370727539 time for calcul the mask position with numpy : 0.021759748458862305 nb_pixel_total : 41048 time to create 1 rle with old method : 0.04727935791015625 time for calcul the mask position with numpy : 0.021472930908203125 nb_pixel_total : 30927 time to create 1 rle with old method : 0.03427267074584961 create new chi : 1.378814935684204 time to delete rle : 0.0012707710266113281 batch 1 Loaded 19 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 6686 TO DO : save crop sub photo not yet done ! save time : 0.42514705657958984 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 4.1891679763793945 time for calcul the mask position with numpy : 0.6748085021972656 nb_pixel_total : 6696441 time to create 1 rle with new method : 1.0736031532287598 time for calcul the mask position with numpy : 0.0358273983001709 nb_pixel_total : 12227 time to create 1 rle with old method : 0.013783931732177734 time for calcul the mask position with numpy : 0.036357879638671875 nb_pixel_total : 114047 time to create 1 rle with old method : 0.12578821182250977 time for calcul the mask position with numpy : 0.033521413803100586 nb_pixel_total : 36073 time to create 1 rle with old method : 0.04042387008666992 time for calcul the mask position with numpy : 0.03306746482849121 nb_pixel_total : 18840 time to create 1 rle with old method : 0.02105426788330078 time for calcul the mask position with numpy : 0.022565603256225586 nb_pixel_total : 47284 time to create 1 rle with old method : 0.05256175994873047 time for calcul the mask position with numpy : 0.02185821533203125 nb_pixel_total : 67195 time to create 1 rle with old method : 0.07415199279785156 time for calcul the mask position with numpy : 0.022376298904418945 nb_pixel_total : 16659 time to create 1 rle with old method : 0.018462657928466797 time for calcul the mask position with numpy : 0.02267765998840332 nb_pixel_total : 11035 time to create 1 rle with old method : 0.01230001449584961 time for calcul the mask position with numpy : 0.02141571044921875 nb_pixel_total : 30439 time to create 1 rle with old method : 0.03366541862487793 create new chi : 2.4295849800109863 time to delete rle : 0.0010721683502197266 batch 1 Loaded 19 chid ids of type : 3594 +++++++++Number RLEs to save : 6738 TO DO : save crop sub photo not yet done ! save time : 0.43939876556396484 nb_obj : 15 nb_hashtags : 3 time to prepare the origin masks : 6.427402973175049 time for calcul the mask position with numpy : 0.568225622177124 nb_pixel_total : 6354476 time to create 1 rle with new method : 0.9814169406890869 time for calcul the mask position with numpy : 0.03947734832763672 nb_pixel_total : 60302 time to create 1 rle with old method : 0.08109569549560547 time for calcul the mask position with numpy : 0.03587150573730469 nb_pixel_total : 74125 time to create 1 rle with old method : 0.08860635757446289 time for calcul the mask position with numpy : 0.03539919853210449 nb_pixel_total : 74261 time to create 1 rle with old method : 0.08086967468261719 time for calcul the mask position with numpy : 0.03523111343383789 nb_pixel_total : 15383 time to create 1 rle with old method : 0.017174243927001953 time for calcul the mask position with numpy : 0.03536844253540039 nb_pixel_total : 62191 time to create 1 rle with old method : 0.07513999938964844 time for calcul the mask position with numpy : 0.03886604309082031 nb_pixel_total : 9520 time to create 1 rle with old method : 0.01086282730102539 time for calcul the mask position with numpy : 0.03526616096496582 nb_pixel_total : 40976 time to create 1 rle with old method : 0.04765796661376953 time for calcul the mask position with numpy : 0.03927946090698242 nb_pixel_total : 39366 time to create 1 rle with old method : 0.05360889434814453 time for calcul the mask position with numpy : 0.022171497344970703 nb_pixel_total : 28640 time to create 1 rle with old method : 0.03191733360290527 time for calcul the mask position with numpy : 0.021924495697021484 nb_pixel_total : 38002 time to create 1 rle with old method : 0.04197812080383301 time for calcul the mask position with numpy : 0.024076461791992188 nb_pixel_total : 28732 time to create 1 rle with old method : 0.03225088119506836 time for calcul the mask position with numpy : 0.022987842559814453 nb_pixel_total : 66669 time to create 1 rle with old method : 0.07532715797424316 time for calcul the mask position with numpy : 0.023463964462280273 nb_pixel_total : 58445 time to create 1 rle with old method : 0.06660914421081543 time for calcul the mask position with numpy : 0.024031877517700195 nb_pixel_total : 26671 time to create 1 rle with old method : 0.03463125228881836 time for calcul the mask position with numpy : 0.02305889129638672 nb_pixel_total : 72481 time to create 1 rle with old method : 0.08552885055541992 create new chi : 2.8724184036254883 time to delete rle : 0.0019528865814208984 batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++++++Number RLEs to save : 11140 TO DO : save crop sub photo not yet done ! save time : 0.7140591144561768 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 4.877098798751831 time for calcul the mask position with numpy : 0.4823780059814453 nb_pixel_total : 6657074 time to create 1 rle with new method : 0.9192318916320801 time for calcul the mask position with numpy : 0.035924673080444336 nb_pixel_total : 5689 time to create 1 rle with old method : 0.006415128707885742 time for calcul the mask position with numpy : 0.03544497489929199 nb_pixel_total : 16532 time to create 1 rle with old method : 0.018501758575439453 time for calcul the mask position with numpy : 0.04027152061462402 nb_pixel_total : 162913 time to create 1 rle with new method : 0.7820072174072266 time for calcul the mask position with numpy : 0.025540590286254883 nb_pixel_total : 20769 time to create 1 rle with old method : 0.023295164108276367 time for calcul the mask position with numpy : 0.03557944297790527 nb_pixel_total : 8815 time to create 1 rle with old method : 0.009950876235961914 time for calcul the mask position with numpy : 0.03747987747192383 nb_pixel_total : 107433 time to create 1 rle with old method : 0.1418623924255371 time for calcul the mask position with numpy : 0.03506350517272949 nb_pixel_total : 29078 time to create 1 rle with old method : 0.03242969512939453 time for calcul the mask position with numpy : 0.03392481803894043 nb_pixel_total : 41937 time to create 1 rle with old method : 0.04648900032043457 create new chi : 2.808619260787964 time to delete rle : 0.0008904933929443359 batch 1 Loaded 17 chid ids of type : 3594 ++++++++++Number RLEs to save : 5438 TO DO : save crop sub photo not yet done ! save time : 0.3498857021331787 map_output_result : {1361633751: (0.0, 'Should be the crop_list due to order', 0), 1361633240: (0.0, 'Should be the crop_list due to order', 0), 1361633234: (0.0, 'Should be the crop_list due to order', 0), 1361633174: (0.0, 'Should be the crop_list due to order', 0), 1361633171: (0.0, 'Should be the crop_list due to order', 0), 1361633166: (0.0, 'Should be the crop_list due to order', 0), 1361633164: (0.0, 'Should be the crop_list due to order', 0), 1361633161: (0.0, 'Should be the crop_list due to order', 0), 1361633155: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1361633751, 1361633240, 1361633234, 1361633174, 1361633171, 1361633166, 1361633164, 1361633161, 1361633155] Looping around the photos to save general results len do output : 9 /1361633751.Didn't retrieve data . /1361633240.Didn't retrieve data . /1361633234.Didn't retrieve data . /1361633174.Didn't retrieve data . /1361633171.Didn't retrieve data . /1361633166.Didn't retrieve data . /1361633164.Didn't retrieve data . /1361633161.Didn't retrieve data . /1361633155.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633751', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633240', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633234', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633174', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633171', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633166', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633164', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633161', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633155', None, None, None, None, None, '2984097') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.013566732406616211 save_final save missing photos in datou_result : time spend for datou_step_exec : 74.13298010826111 time spend to save output : 0.014056205749511719 total time spend for step 3 : 74.14703631401062 step4:ventilate_hashtags_in_portfolio Thu May 29 17:53:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23441275 get user id for portfolio 23441275 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23441275 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','metal','pet_fonce','environnement','mal_croppe','autre','carton','papier','pehd','flou','pet_clair')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23441275 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','metal','pet_fonce','environnement','mal_croppe','autre','carton','papier','pehd','flou','pet_clair')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23441275 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','metal','pet_fonce','environnement','mal_croppe','autre','carton','papier','pehd','flou','pet_clair')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23441358,23441359,23441360,23441361,23441362,23441363,23441364,23441365,23441366,23441367,23441368?tags=background,metal,pet_fonce,environnement,mal_croppe,autre,carton,papier,pehd,flou,pet_clair Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1361633751, 1361633240, 1361633234, 1361633174, 1361633171, 1361633166, 1361633164, 1361633161, 1361633155] Looping around the photos to save general results len do output : 1 /23441275. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633751', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633240', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633234', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633174', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633171', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633166', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633164', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633161', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633155', None, None, None, None, None, '2984097') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013645172119140625 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5447616577148438 time spend to save output : 0.01399087905883789 total time spend for step 4 : 1.5587525367736816 step5:final Thu May 29 17:53:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1361633751: ('0.10531715547587349',), 1361633240: ('0.10531715547587349',), 1361633234: ('0.10531715547587349',), 1361633174: ('0.10531715547587349',), 1361633171: ('0.10531715547587349',), 1361633166: ('0.10531715547587349',), 1361633164: ('0.10531715547587349',), 1361633161: ('0.10531715547587349',), 1361633155: ('0.10531715547587349',)} new output for save of step final : {1361633751: ('0.10531715547587349',), 1361633240: ('0.10531715547587349',), 1361633234: ('0.10531715547587349',), 1361633174: ('0.10531715547587349',), 1361633171: ('0.10531715547587349',), 1361633166: ('0.10531715547587349',), 1361633164: ('0.10531715547587349',), 1361633161: ('0.10531715547587349',), 1361633155: ('0.10531715547587349',)} [1361633751, 1361633240, 1361633234, 1361633174, 1361633171, 1361633166, 1361633164, 1361633161, 1361633155] Looping around the photos to save general results len do output : 9 /1361633751.Didn't retrieve data . /1361633240.Didn't retrieve data . /1361633234.Didn't retrieve data . /1361633174.Didn't retrieve data . /1361633171.Didn't retrieve data . /1361633166.Didn't retrieve data . /1361633164.Didn't retrieve data . /1361633161.Didn't retrieve data . /1361633155.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633751', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633240', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633234', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633174', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633171', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633166', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633164', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633161', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633155', None, None, None, None, None, '2984097') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.013581514358520508 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1042320728302002 time spend to save output : 0.014022111892700195 total time spend for step 5 : 0.11825418472290039 step6:blur_detection Thu May 29 17:53:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141.jpg resize: (2160, 3264) 1361633751 -3.7899511118484073 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684.jpg resize: (2160, 3264) 1361633240 -2.348416565355306 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1.jpg resize: (2160, 3264) 1361633234 -1.061898536487279 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b.jpg resize: (2160, 3264) 1361633174 -2.8921724025381437 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1.jpg resize: (2160, 3264) 1361633171 3.1781373591531623 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a.jpg resize: (2160, 3264) 1361633166 -1.3272413411327768 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43.jpg resize: (2160, 3264) 1361633164 -2.281002559610725 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86.jpg resize: (2160, 3264) 1361633161 -2.611704330612014 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03.jpg resize: (2160, 3264) 1361633155 -1.8706113096338257 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306902_0.png resize: (292, 225) 1361650044 -2.076340824866227 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306905_0.png resize: (243, 168) 1361650045 -0.3395094007895753 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306911_0.png resize: (289, 402) 1361650046 -3.024023995531959 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306915_0.png resize: (813, 784) 1361650047 -2.335063566867124 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306917_0.png resize: (287, 159) 1361650048 -1.4135747118909567 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306920_0.png resize: (271, 193) 1361650049 -1.5836503880854957 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306924_0.png resize: (205, 115) 1361650050 1.1039259555206875 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306927_0.png resize: (214, 71) 1361650051 -1.4034331861832379 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306931_0.png resize: (1228, 818) 1361650052 -2.6750897593471206 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306935_0.png resize: (570, 568) 1361650053 -1.0550513085402398 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306937_0.png resize: (144, 152) 1361650054 -0.3976877815999056 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306940_0.png resize: (205, 80) 1361650055 0.5672094052003533 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306941_0.png resize: (219, 403) 1361650056 -2.0283885658704284 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306942_0.png resize: (1055, 865) 1361650057 -0.6611505117542987 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306943_0.png resize: (250, 283) 1361650058 -2.409141857429077 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306944_0.png resize: (163, 308) 1361650059 -2.155860695379227 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306946_0.png resize: (203, 144) 1361650060 -1.5791151783949204 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306947_0.png resize: (160, 122) 1361650061 -1.4775914498182199 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306948_0.png resize: (84, 205) 1361650062 -2.0341625128614496 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306951_0.png resize: (119, 72) 1361650063 -1.2401425413592562 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306952_0.png resize: (233, 241) 1361650064 -1.1193817020696295 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306954_0.png resize: (270, 195) 1361650065 0.2593069691272608 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306956_0.png resize: (257, 239) 1361650066 0.350539130698411 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306957_0.png resize: (196, 263) 1361650067 -0.48113110657203423 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306959_0.png resize: (231, 208) 1361650068 -1.3374296276887228 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306960_0.png resize: (291, 291) 1361650069 -1.6052585733348916 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306962_0.png resize: (220, 118) 1361650070 -1.840474767177357 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306964_0.png resize: (174, 229) 1361650071 -0.7484580354131906 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306966_0.png resize: (166, 143) 1361650072 -1.1515108972143253 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306968_0.png resize: (228, 289) 1361650073 -1.4600249045489753 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306973_0.png resize: (187, 167) 1361650074 -1.0221936127757574 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306977_0.png resize: (269, 401) 1361650075 -2.527391903888561 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306978_0.png resize: (207, 170) 1361650076 -0.6574279777098148 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306979_0.png resize: (272, 316) 1361650077 -2.1156431410234515 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306980_0.png resize: (341, 326) 1361650078 -1.3439559233952831 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306981_0.png resize: (225, 188) 1361650079 -0.6848538231438039 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306983_0.png resize: (171, 241) 1361650080 -1.853314957696201 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306984_0.png resize: (213, 280) 1361650081 -0.7889753487656069 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306987_0.png resize: (449, 229) 1361650082 -1.574740315379676 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306988_0.png resize: (108, 173) 1361650083 -0.011680357242946188 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306991_0.png resize: (232, 361) 1361650084 -0.524610904861418 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306992_0.png resize: (306, 230) 1361650085 0.012128480411272805 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306993_0.png resize: (205, 221) 1361650086 -1.529933412074049 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306995_0.png resize: (121, 103) 1361650087 -0.9856414861845927 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306998_0.png resize: (118, 172) 1361650088 1.5330562563163925 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306907_0.png resize: (266, 390) 1361650098 -2.2681918994049868 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306909_0.png resize: (808, 892) 1361650099 -2.8443957292350657 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306912_0.png resize: (352, 335) 1361650100 -1.0542241804611894 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306913_0.png resize: (257, 381) 1361650101 -2.9997560044798828 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306914_0.png resize: (355, 322) 1361650102 -2.166955487077243 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306916_0.png resize: (836, 977) 1361650103 -1.8220006873892793 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306922_0.png resize: (298, 455) 1361650104 -1.0380935533612483 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306923_0.png resize: (149, 187) 1361650105 -2.4944733228167557 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306929_0.png resize: (355, 332) 1361650106 -1.7582237721617364 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306932_0.png resize: (275, 750) 1361650107 -2.4403664369277513 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306934_0.png resize: (166, 115) 1361650108 0.07973835748834573 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306965_0.png resize: (248, 516) 1361650109 -0.841761504159869 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306969_0.png resize: (161, 107) 1361650110 -1.3050474905840035 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306970_0.png resize: (151, 184) 1361650111 -1.4968899191778005 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306972_0.png resize: (291, 267) 1361650112 -1.2546220761912878 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306976_0.png resize: (85, 198) 1361650113 -0.3043767669724302 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306986_0.png resize: (104, 164) 1361650114 -0.9614350347032605 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306996_0.png resize: (107, 230) 1361650115 0.9076428087790062 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306903_0.png resize: (468, 353) 1361650141 -3.0978922641088693 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306904_0.png resize: (165, 334) 1361650143 -3.6325267135183577 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306906_0.png resize: (243, 408) 1361650144 -3.1399821437966917 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306908_0.png resize: (121, 186) 1361650145 -2.2792718354305976 treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306910_0.png resize: (681, 677) 1361650146 -2.4045290274895774 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306918_0.png resize: (177, 427) 1361650147 -2.810077420512115 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306919_0.png resize: (257, 367) 1361650148 -2.0697176897412977 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306921_0.png resize: (104, 152) 1361650149 -0.7188129247974244 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306925_0.png resize: (325, 344) 1361650150 -2.80540142962551 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306926_0.png resize: (375, 291) 1361650151 -0.9712419492493646 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306928_0.png resize: (214, 194) 1361650152 -1.4173316963939955 treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306930_0.png resize: (198, 260) 1361650153 -1.4280459898174795 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306933_0.png resize: (264, 245) 1361650154 -1.076845735471751 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306936_0.png resize: (411, 1023) 1361650155 -0.8508382524799084 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306938_0.png resize: (448, 569) 1361650156 -1.306806758918147 treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306939_0.png resize: (99, 338) 1361650158 -1.2895415655935196 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306945_0.png resize: (364, 231) 1361650159 -2.114998945369876 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306950_0.png resize: (115, 191) 1361650160 -2.673778012599045 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306953_0.png resize: (716, 858) 1361650161 -0.494405680252931 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306955_0.png resize: (249, 613) 1361650162 -0.49638071906896974 treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306958_0.png resize: (89, 158) 1361650163 0.30258603198802175 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306961_0.png resize: (202, 124) 1361650164 -2.6034916755110915 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306963_0.png resize: (283, 382) 1361650165 -2.2199762395267797 treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306967_0.png resize: (89, 214) 1361650166 0.09097689600686144 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306971_0.png resize: (311, 315) 1361650167 -1.7262099283731978 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306974_0.png resize: (361, 138) 1361650168 -2.6610484546529545 treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306975_0.png resize: (504, 403) 1361650169 -1.5784183974004844 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306982_0.png resize: (204, 268) 1361650170 -1.6924205804798456 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306985_0.png resize: (330, 396) 1361650171 -1.473279096973311 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306989_0.png resize: (499, 194) 1361650172 -2.2418597159325016 treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306990_0.png resize: (734, 137) 1361650173 -1.1115341130288998 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306997_0.png resize: (292, 802) 1361650174 -0.5222778286645844 treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306949_0.png resize: (266, 190) 1361650178 -0.7971575483623188 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306994_0.png resize: (333, 451) 1361650179 -1.1387280492116183 treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306999_0.png resize: (87, 74) 1361650180 20.0 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 107 time used for this insertion : 0.017093181610107422 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 107 time used for this insertion : 0.027892589569091797 save missing photos in datou_result : time spend for datou_step_exec : 36.860164403915405 time spend to save output : 0.05086183547973633 total time spend for step 6 : 36.91102623939514 step7:brightness Thu May 29 17:54:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141.jpg treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684.jpg treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1.jpg treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b.jpg treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1.jpg treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a.jpg treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43.jpg treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86.jpg treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03.jpg treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306902_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306905_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306911_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306915_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306917_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306920_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306924_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306927_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306931_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306935_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306937_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306940_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306941_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306942_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306943_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306944_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306946_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306947_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306948_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306951_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306952_0.png treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306954_0.png treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306956_0.png treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306957_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306959_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306960_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306962_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306964_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306966_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306968_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306973_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306977_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306978_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306979_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306980_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306981_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306983_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306984_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306987_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306988_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306991_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306992_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306993_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306995_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306998_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306907_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306909_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306912_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306913_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306914_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306916_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306922_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306923_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306929_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306932_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306934_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306965_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306969_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306970_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306972_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306976_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306986_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306996_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306903_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306904_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306906_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306908_0.png treat image : temp/1748533829_3297538_1361633751_517d4087546a2ee0cbf726fb08599141_rle_crop_3815306910_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306918_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306919_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306921_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306925_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306926_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306928_0.png treat image : temp/1748533829_3297538_1361633240_0233da6939d2e1d9dbb6f69b6c52a684_rle_crop_3815306930_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306933_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306936_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306938_0.png treat image : temp/1748533829_3297538_1361633234_ae02d030b438735710c13c7bf51e15c1_rle_crop_3815306939_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306945_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306950_0.png treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306953_0.png treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306955_0.png treat image : temp/1748533829_3297538_1361633171_8234d5c9b6cb525cf7d6f856ee2efda1_rle_crop_3815306958_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306961_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306963_0.png treat image : temp/1748533829_3297538_1361633166_baf204c56ea87d90b45c018161acfc0a_rle_crop_3815306967_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306971_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306974_0.png treat image : temp/1748533829_3297538_1361633164_f9a24b4af3cc80893c2ac38052bd4a43_rle_crop_3815306975_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306982_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306985_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306989_0.png treat image : temp/1748533829_3297538_1361633161_9971757c08311b3f9cfea90bcc2e5e86_rle_crop_3815306990_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306997_0.png treat image : temp/1748533829_3297538_1361633174_b66d89c2bcaced37eb68a38fc9f8a83b_rle_crop_3815306949_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306994_0.png treat image : temp/1748533829_3297538_1361633155_d57d129b8d0459bdd83477fb5f8d1d03_rle_crop_3815306999_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 107 time used for this insertion : 0.017407655715942383 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 107 time used for this insertion : 0.02779364585876465 save missing photos in datou_result : time spend for datou_step_exec : 8.502290487289429 time spend to save output : 0.05040931701660156 total time spend for step 7 : 8.55269980430603 step8:velours_tree Thu May 29 17:54:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.16330933570861816 time spend to save output : 5.316734313964844e-05 total time spend for step 8 : 0.1633625030517578 step9:send_mail_cod Thu May 29 17:54:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23441275_29-05-2025_17_54_44.pdf 23441358 imagette234413581748534084 23441359 imagette234413591748534084 23441360 imagette234413601748534084 23441362 imagette234413621748534084 23441363 change filename to text .change filename to text .change filename to text .imagette234413631748534084 23441364 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234413641748534085 23441365 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234413651748534086 23441366 imagette234413661748534087 23441367 imagette234413671748534087 23441368 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234413681748534087 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23441275 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23441358,23441359,23441360,23441361,23441362,23441363,23441364,23441365,23441366,23441367,23441368?tags=background,metal,pet_fonce,environnement,mal_croppe,autre,carton,papier,pehd,flou,pet_clair args[1361633751] : ((1361633751, -3.7899511118484073, 492609224), (1361633751, 0.04367971779704661, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633240] : ((1361633240, -2.348416565355306, 492609224), (1361633240, 0.050582394904223685, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633234] : ((1361633234, -1.061898536487279, 492688767), (1361633234, 0.49725475511704087, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633174] : ((1361633174, -2.8921724025381437, 492609224), (1361633174, 0.32672173382659997, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633171] : ((1361633171, 3.1781373591531623, 492688767), (1361633171, 0.07493204351985598, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633166] : ((1361633166, -1.3272413411327768, 492688767), (1361633166, 0.09990397842986642, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633164] : ((1361633164, -2.281002559610725, 492609224), (1361633164, 0.1196245956609772, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633161] : ((1361633161, -2.611704330612014, 492609224), (1361633161, 0.2561182234750154, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com args[1361633155] : ((1361633155, -1.8706113096338257, 492688767), (1361633155, 0.28771048511550573, 2107752395), '0.10531715547587349') We are sending mail with results at report@fotonower.com refus_total : 0.10531715547587349 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23441275 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23441275_29-05-2025_17_54_44.pdf results_Auto_P23441275_29-05-2025_17_54_44.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23441275_29-05-2025_17_54_44.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23441275','results_Auto_P23441275_29-05-2025_17_54_44.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23441275_29-05-2025_17_54_44.pdf','pdf','','0.74','0.10531715547587349') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23441275

https://www.fotonower.com/image?json=false&list_photos_id=1361633751
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633240
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633234
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633174
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633171
La photo est trop floue, merci de reprendre une photo.(avec le score = 3.1781373591531623)
https://www.fotonower.com/image?json=false&list_photos_id=1361633166
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633164
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633161
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361633155
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 10.53%
Veuillez trouver les photos des contaminants.

exemples de contaminants: autre: https://www.fotonower.com/view/23441363?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23441364?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23441365?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23441368?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23441275_29-05-2025_17_54_44.pdf.

Lien vers velours :https://www.fotonower.com/velours/23441358,23441359,23441360,23441361,23441362,23441363,23441364,23441365,23441366,23441367,23441368?tags=background,metal,pet_fonce,environnement,mal_croppe,autre,carton,papier,pehd,flou,pet_clair.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 29 May 2025 15:54:52 GMT Content-Length: 0 Connection: close X-Message-Id: 7F4Gi3TaQOGOyPzjF0qehw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1361633751, 1361633240, 1361633234, 1361633174, 1361633171, 1361633166, 1361633164, 1361633161, 1361633155] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633751', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633240', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633234', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633174', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633171', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633166', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633164', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633161', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633155', None, None, None, None, None, '2984097') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.012965679168701172 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.253936767578125 time spend to save output : 0.01327371597290039 total time spend for step 9 : 7.267210483551025 step10:split_time_score Thu May 29 17:54:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 9),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 29052025 23441275 Nombre de photos uploadées : 9 / 23040 (0%) 29052025 23441275 Nombre de photos taguées (types de déchets): 0 / 9 (0%) 29052025 23441275 Nombre de photos taguées (volume) : 0 / 9 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 7.3909759521484375e-06 ????????? elapsed_time : fill_and_build_computed_from_old_data 0.0006911754608154297 elapsed_time : insert_dashboard_record_day_entry 0.0227510929107666 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23440003 order by id desc limit 1 Qualite : 0.13914992019184222 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23440005_29-05-2025_14_36_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23440005 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23440005 AND mptpi.`type`=3594 To do Qualite : 0.09789808573892522 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439680_29-05-2025_13_14_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23439680 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23439680 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23440235 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23440236 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23441274 order by id desc limit 1 Qualite : 0.10531715547587349 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23441275_29-05-2025_17_54_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23441275 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23441275 AND mptpi.`type`=3594 To do Qualite : 0.11244704080995825 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23441003_29-05-2025_17_24_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23441003 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23441003 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23441279 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'29052025': {'nb_upload': 9, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1361633751, 1361633240, 1361633234, 1361633174, 1361633171, 1361633166, 1361633164, 1361633161, 1361633155] Looping around the photos to save general results len do output : 1 /23441275Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633751', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633240', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633234', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633174', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633171', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633166', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633164', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633161', None, None, None, None, None, '2984097') ('3318', None, None, None, None, None, None, None, '2984097') ('3318', '23441275', '1361633155', None, None, None, None, None, '2984097') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013399124145507812 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.69573450088501 time spend to save output : 0.013664960861206055 total time spend for step 10 : 4.709399461746216 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 9 set_done_treatment 121.92user 109.63system 4:30.59elapsed 85%CPU (0avgtext+0avgdata 4591948maxresident)k 790040inputs+86264outputs (12304major+10552358minor)pagefaults 0swaps