python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2365930 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2961022'] with mtr_portfolio_ids : ['23360449'] and first list_photo_ids : [] new path : /proc/2365930/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.1939735412597656 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue May 27 01:30:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6810 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-27 01:30:32.451796: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-27 01:30:32.479059: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-27 01:30:32.481347: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7138000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-27 01:30:32.481412: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-27 01:30:32.485548: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-27 01:30:32.752892: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x97bcdd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-27 01:30:32.752944: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-27 01:30:32.754168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-27 01:30:32.755042: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-27 01:30:32.762689: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-27 01:30:32.768339: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-27 01:30:32.769472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-27 01:30:32.778508: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-27 01:30:32.780947: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-27 01:30:32.794069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-27 01:30:32.795723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-27 01:30:32.795817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-27 01:30:32.796683: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-27 01:30:32.796703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-27 01:30:32.796716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-27 01:30:32.798197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6259 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-27 01:30:33.052263: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-27 01:30:33.052381: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-27 01:30:33.052402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-27 01:30:33.052420: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-27 01:30:33.052439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-27 01:30:33.052456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-27 01:30:33.052474: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-27 01:30:33.052492: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-27 01:30:33.053756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-27 01:30:33.054975: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-27 01:30:33.055014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-27 01:30:33.055032: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-27 01:30:33.055049: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-27 01:30:33.055065: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-27 01:30:33.055082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-27 01:30:33.055098: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-27 01:30:33.055115: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-27 01:30:33.056185: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-27 01:30:33.056218: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-27 01:30:33.056226: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-27 01:30:33.056233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-27 01:30:33.057211: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6259 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-27 01:30:41.721949: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-27 01:30:41.909224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 15.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 Detection mask done ! Trying to reset tf kernel 2366485 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1521 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6810 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.033594369888305664 nb_pixel_total : 665417 time to create 1 rle with new method : 0.06113314628601074 length of segment : 748 time for calcul the mask position with numpy : 0.006741762161254883 nb_pixel_total : 326815 time to create 1 rle with new method : 0.05649709701538086 length of segment : 1101 time for calcul the mask position with numpy : 0.008927106857299805 nb_pixel_total : 406913 time to create 1 rle with new method : 0.04309892654418945 length of segment : 757 time for calcul the mask position with numpy : 0.0037152767181396484 nb_pixel_total : 288845 time to create 1 rle with new method : 0.013695955276489258 length of segment : 504 time for calcul the mask position with numpy : 0.0042572021484375 nb_pixel_total : 296724 time to create 1 rle with new method : 0.016993045806884766 length of segment : 1107 time for calcul the mask position with numpy : 0.003968477249145508 nb_pixel_total : 293071 time to create 1 rle with new method : 0.016941070556640625 length of segment : 704 time for calcul the mask position with numpy : 0.001138448715209961 nb_pixel_total : 87877 time to create 1 rle with old method : 0.09893536567687988 length of segment : 645 time for calcul the mask position with numpy : 0.0006153583526611328 nb_pixel_total : 24665 time to create 1 rle with old method : 0.0286104679107666 length of segment : 252 time for calcul the mask position with numpy : 0.0006017684936523438 nb_pixel_total : 29599 time to create 1 rle with old method : 0.03350234031677246 length of segment : 527 time for calcul the mask position with numpy : 0.0006968975067138672 nb_pixel_total : 42484 time to create 1 rle with old method : 0.04957175254821777 length of segment : 210 time for calcul the mask position with numpy : 0.0007715225219726562 nb_pixel_total : 49716 time to create 1 rle with old method : 0.056939125061035156 length of segment : 291 time for calcul the mask position with numpy : 0.00048542022705078125 nb_pixel_total : 31271 time to create 1 rle with old method : 0.037102460861206055 length of segment : 142 time for calcul the mask position with numpy : 0.0003209114074707031 nb_pixel_total : 15161 time to create 1 rle with old method : 0.018529415130615234 length of segment : 132 time for calcul the mask position with numpy : 0.00040841102600097656 nb_pixel_total : 28327 time to create 1 rle with old method : 0.037117719650268555 length of segment : 179 time for calcul the mask position with numpy : 0.004402637481689453 nb_pixel_total : 217508 time to create 1 rle with new method : 0.011304140090942383 length of segment : 1320 time for calcul the mask position with numpy : 0.010928630828857422 nb_pixel_total : 337180 time to create 1 rle with new method : 0.053876399993896484 length of segment : 816 time for calcul the mask position with numpy : 0.00035881996154785156 nb_pixel_total : 7401 time to create 1 rle with old method : 0.008332014083862305 length of segment : 98 time for calcul the mask position with numpy : 0.0015439987182617188 nb_pixel_total : 71482 time to create 1 rle with old method : 0.0890498161315918 length of segment : 319 time for calcul the mask position with numpy : 0.0028128623962402344 nb_pixel_total : 156180 time to create 1 rle with new method : 0.0065190792083740234 length of segment : 605 time for calcul the mask position with numpy : 0.0004298686981201172 nb_pixel_total : 15620 time to create 1 rle with old method : 0.02601790428161621 length of segment : 208 time for calcul the mask position with numpy : 0.009718656539916992 nb_pixel_total : 513597 time to create 1 rle with new method : 0.03875303268432617 length of segment : 813 time for calcul the mask position with numpy : 0.001991748809814453 nb_pixel_total : 67765 time to create 1 rle with old method : 0.07478761672973633 length of segment : 489 time for calcul the mask position with numpy : 0.0014157295227050781 nb_pixel_total : 70924 time to create 1 rle with old method : 0.0804297924041748 length of segment : 314 time for calcul the mask position with numpy : 0.0014967918395996094 nb_pixel_total : 66487 time to create 1 rle with old method : 0.07470273971557617 length of segment : 220 time for calcul the mask position with numpy : 0.0010063648223876953 nb_pixel_total : 34898 time to create 1 rle with old method : 0.04085373878479004 length of segment : 180 time for calcul the mask position with numpy : 0.0005843639373779297 nb_pixel_total : 14323 time to create 1 rle with old method : 0.01661396026611328 length of segment : 183 time for calcul the mask position with numpy : 0.0003895759582519531 nb_pixel_total : 10170 time to create 1 rle with old method : 0.012182235717773438 length of segment : 114 time for calcul the mask position with numpy : 0.002318143844604492 nb_pixel_total : 43670 time to create 1 rle with old method : 0.05009889602661133 length of segment : 335 time for calcul the mask position with numpy : 0.0009419918060302734 nb_pixel_total : 27611 time to create 1 rle with old method : 0.030482053756713867 length of segment : 197 time for calcul the mask position with numpy : 0.0005555152893066406 nb_pixel_total : 17005 time to create 1 rle with old method : 0.01960921287536621 length of segment : 151 time for calcul the mask position with numpy : 0.0003571510314941406 nb_pixel_total : 10433 time to create 1 rle with old method : 0.011873722076416016 length of segment : 96 time for calcul the mask position with numpy : 0.0011610984802246094 nb_pixel_total : 29425 time to create 1 rle with old method : 0.032242774963378906 length of segment : 287 time for calcul the mask position with numpy : 0.0006537437438964844 nb_pixel_total : 15889 time to create 1 rle with old method : 0.017490148544311523 length of segment : 171 time for calcul the mask position with numpy : 0.0005104541778564453 nb_pixel_total : 12084 time to create 1 rle with old method : 0.013791084289550781 length of segment : 144 time for calcul the mask position with numpy : 0.001371145248413086 nb_pixel_total : 58328 time to create 1 rle with old method : 0.06525778770446777 length of segment : 283 time for calcul the mask position with numpy : 0.0004074573516845703 nb_pixel_total : 9056 time to create 1 rle with old method : 0.010498285293579102 length of segment : 129 time for calcul the mask position with numpy : 0.0002579689025878906 nb_pixel_total : 12921 time to create 1 rle with old method : 0.01616501808166504 length of segment : 148 time for calcul the mask position with numpy : 0.0003273487091064453 nb_pixel_total : 11203 time to create 1 rle with old method : 0.013511419296264648 length of segment : 76 time for calcul the mask position with numpy : 0.00032782554626464844 nb_pixel_total : 11818 time to create 1 rle with old method : 0.014271259307861328 length of segment : 174 time for calcul the mask position with numpy : 0.0011134147644042969 nb_pixel_total : 31885 time to create 1 rle with old method : 0.037156105041503906 length of segment : 217 time for calcul the mask position with numpy : 0.0007214546203613281 nb_pixel_total : 30212 time to create 1 rle with old method : 0.03527402877807617 length of segment : 179 time for calcul the mask position with numpy : 0.000354766845703125 nb_pixel_total : 9929 time to create 1 rle with old method : 0.011202573776245117 length of segment : 186 time for calcul the mask position with numpy : 0.0008280277252197266 nb_pixel_total : 23656 time to create 1 rle with old method : 0.026248931884765625 length of segment : 184 time for calcul the mask position with numpy : 0.0005424022674560547 nb_pixel_total : 19251 time to create 1 rle with old method : 0.021683454513549805 length of segment : 130 time for calcul the mask position with numpy : 0.000499725341796875 nb_pixel_total : 16220 time to create 1 rle with old method : 0.018519163131713867 length of segment : 143 time for calcul the mask position with numpy : 0.00037479400634765625 nb_pixel_total : 8922 time to create 1 rle with old method : 0.010161876678466797 length of segment : 148 time for calcul the mask position with numpy : 0.0004475116729736328 nb_pixel_total : 14260 time to create 1 rle with old method : 0.015788793563842773 length of segment : 139 time for calcul the mask position with numpy : 0.00018858909606933594 nb_pixel_total : 5973 time to create 1 rle with old method : 0.006966590881347656 length of segment : 95 time for calcul the mask position with numpy : 0.0004036426544189453 nb_pixel_total : 13504 time to create 1 rle with old method : 0.014938592910766602 length of segment : 131 time for calcul the mask position with numpy : 0.0005090236663818359 nb_pixel_total : 10556 time to create 1 rle with old method : 0.012458324432373047 length of segment : 176 time for calcul the mask position with numpy : 0.0013282299041748047 nb_pixel_total : 46112 time to create 1 rle with old method : 0.052741050720214844 length of segment : 251 time for calcul the mask position with numpy : 0.001463174819946289 nb_pixel_total : 57894 time to create 1 rle with old method : 0.06584453582763672 length of segment : 266 time for calcul the mask position with numpy : 0.0014617443084716797 nb_pixel_total : 60652 time to create 1 rle with old method : 0.06840777397155762 length of segment : 278 time for calcul the mask position with numpy : 0.00029921531677246094 nb_pixel_total : 6191 time to create 1 rle with old method : 0.007132530212402344 length of segment : 122 time for calcul the mask position with numpy : 0.0006418228149414062 nb_pixel_total : 23546 time to create 1 rle with old method : 0.026761531829833984 length of segment : 217 time for calcul the mask position with numpy : 0.0007581710815429688 nb_pixel_total : 40591 time to create 1 rle with old method : 0.045969486236572266 length of segment : 242 time for calcul the mask position with numpy : 0.0004944801330566406 nb_pixel_total : 18461 time to create 1 rle with old method : 0.021160125732421875 length of segment : 164 time for calcul the mask position with numpy : 0.0002906322479248047 nb_pixel_total : 8131 time to create 1 rle with old method : 0.00950479507446289 length of segment : 115 time for calcul the mask position with numpy : 0.0046842098236083984 nb_pixel_total : 236020 time to create 1 rle with new method : 0.01062464714050293 length of segment : 918 time for calcul the mask position with numpy : 0.00047135353088378906 nb_pixel_total : 9303 time to create 1 rle with old method : 0.010930776596069336 length of segment : 109 time for calcul the mask position with numpy : 0.0013232231140136719 nb_pixel_total : 49901 time to create 1 rle with old method : 0.05768227577209473 length of segment : 305 time for calcul the mask position with numpy : 0.0005362033843994141 nb_pixel_total : 12709 time to create 1 rle with old method : 0.015061140060424805 length of segment : 143 time for calcul the mask position with numpy : 0.0004429817199707031 nb_pixel_total : 14276 time to create 1 rle with old method : 0.016980648040771484 length of segment : 109 time for calcul the mask position with numpy : 0.0036597251892089844 nb_pixel_total : 145225 time to create 1 rle with old method : 0.16541242599487305 length of segment : 455 time for calcul the mask position with numpy : 0.0021202564239501953 nb_pixel_total : 88534 time to create 1 rle with old method : 0.09919357299804688 length of segment : 326 time for calcul the mask position with numpy : 0.0013098716735839844 nb_pixel_total : 42322 time to create 1 rle with old method : 0.0476231575012207 length of segment : 280 time for calcul the mask position with numpy : 0.002311229705810547 nb_pixel_total : 89400 time to create 1 rle with old method : 0.10106372833251953 length of segment : 276 time for calcul the mask position with numpy : 0.0010657310485839844 nb_pixel_total : 51019 time to create 1 rle with old method : 0.06519770622253418 length of segment : 422 time for calcul the mask position with numpy : 0.0008940696716308594 nb_pixel_total : 43723 time to create 1 rle with old method : 0.0520782470703125 length of segment : 306 time for calcul the mask position with numpy : 0.002158641815185547 nb_pixel_total : 86192 time to create 1 rle with old method : 0.09711813926696777 length of segment : 399 time for calcul the mask position with numpy : 0.004876375198364258 nb_pixel_total : 224147 time to create 1 rle with new method : 0.007884979248046875 length of segment : 431 time for calcul the mask position with numpy : 0.004339933395385742 nb_pixel_total : 151838 time to create 1 rle with new method : 0.007673501968383789 length of segment : 655 time for calcul the mask position with numpy : 0.0009708404541015625 nb_pixel_total : 36961 time to create 1 rle with old method : 0.04307365417480469 length of segment : 351 time for calcul the mask position with numpy : 0.0007617473602294922 nb_pixel_total : 15372 time to create 1 rle with old method : 0.025351762771606445 length of segment : 186 time for calcul the mask position with numpy : 0.0007228851318359375 nb_pixel_total : 19801 time to create 1 rle with old method : 0.023479223251342773 length of segment : 201 time for calcul the mask position with numpy : 0.0004420280456542969 nb_pixel_total : 18728 time to create 1 rle with old method : 0.02167797088623047 length of segment : 229 time for calcul the mask position with numpy : 0.0036902427673339844 nb_pixel_total : 138324 time to create 1 rle with old method : 0.1526479721069336 length of segment : 580 time for calcul the mask position with numpy : 0.0012013912200927734 nb_pixel_total : 62485 time to create 1 rle with old method : 0.07208895683288574 length of segment : 433 time for calcul the mask position with numpy : 0.0010857582092285156 nb_pixel_total : 78337 time to create 1 rle with old method : 0.08945512771606445 length of segment : 363 time for calcul the mask position with numpy : 0.0008289813995361328 nb_pixel_total : 24312 time to create 1 rle with old method : 0.029000520706176758 length of segment : 238 time for calcul the mask position with numpy : 0.0001423358917236328 nb_pixel_total : 6659 time to create 1 rle with old method : 0.00811147689819336 length of segment : 71 time for calcul the mask position with numpy : 0.0005135536193847656 nb_pixel_total : 18714 time to create 1 rle with old method : 0.022181272506713867 length of segment : 142 time for calcul the mask position with numpy : 0.0016546249389648438 nb_pixel_total : 69955 time to create 1 rle with old method : 0.08021020889282227 length of segment : 410 time for calcul the mask position with numpy : 0.0021390914916992188 nb_pixel_total : 67554 time to create 1 rle with old method : 0.07675838470458984 length of segment : 291 time for calcul the mask position with numpy : 0.0013434886932373047 nb_pixel_total : 45263 time to create 1 rle with old method : 0.05068230628967285 length of segment : 406 time for calcul the mask position with numpy : 0.000789642333984375 nb_pixel_total : 24176 time to create 1 rle with old method : 0.027949094772338867 length of segment : 264 time for calcul the mask position with numpy : 0.005978107452392578 nb_pixel_total : 293961 time to create 1 rle with new method : 0.013526201248168945 length of segment : 756 time for calcul the mask position with numpy : 0.001096963882446289 nb_pixel_total : 41919 time to create 1 rle with old method : 0.04822874069213867 length of segment : 303 time for calcul the mask position with numpy : 0.0005092620849609375 nb_pixel_total : 13955 time to create 1 rle with old method : 0.017055273056030273 length of segment : 134 time for calcul the mask position with numpy : 0.0003337860107421875 nb_pixel_total : 6330 time to create 1 rle with old method : 0.0077762603759765625 length of segment : 121 time spent for convertir_results : 9.120567321777344 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 90 chid ids of type : 3594 Number RLEs to save : 29165 save missing photos in datou_result : time spend for datou_step_exec : 47.26538133621216 time spend to save output : 4.762491703033447 total time spend for step 1 : 52.027873039245605 step2:crop_condition Tue May 27 01:31:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 90 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 59 About to insert : list_path_to_insert length 59 new photo from crops ! About to upload 59 photos upload in portfolio : 3736932 init cache_photo without model_param we have 59 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748302302_2365930 we have uploaded 59 photos in the portfolio 3736932 time of upload the photos Elapsed time : 13.494972467422485 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3736932 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748302321_2365930 we have uploaded 8 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.335059404373169 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3736932 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748302331_2365930 we have uploaded 21 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.863117456436157 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748302339_2365930 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7257909774780273 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748302341_2365930 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5057029724121094 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1360648563, 1360648521, 1360623490, 1360623484, 1360623480, 1360623466, 1360623387, 1360621971] Looping around the photos to save general results len do output : 90 /1361191094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361191497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648563', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648521', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623490', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623484', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623480', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623466', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623387', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360621971', None, None, None, None, None, '2961022') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 278 time used for this insertion : 0.08730769157409668 save_final save missing photos in datou_result : time spend for datou_step_exec : 60.40367674827576 time spend to save output : 0.09069061279296875 total time spend for step 2 : 60.494367361068726 step3:rle_unique_nms_with_priority Tue May 27 01:32:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 90 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 3.9787282943725586 time for calcul the mask position with numpy : 0.2035350799560547 nb_pixel_total : 4464051 time to create 1 rle with new method : 0.4363136291503906 time for calcul the mask position with numpy : 0.020751476287841797 nb_pixel_total : 28327 time to create 1 rle with old method : 0.032758235931396484 time for calcul the mask position with numpy : 0.02083444595336914 nb_pixel_total : 15161 time to create 1 rle with old method : 0.017138004302978516 time for calcul the mask position with numpy : 0.020048141479492188 nb_pixel_total : 31271 time to create 1 rle with old method : 0.03496861457824707 time for calcul the mask position with numpy : 0.020539283752441406 nb_pixel_total : 49716 time to create 1 rle with old method : 0.059729576110839844 time for calcul the mask position with numpy : 0.03568387031555176 nb_pixel_total : 42484 time to create 1 rle with old method : 0.04862380027770996 time for calcul the mask position with numpy : 0.035364389419555664 nb_pixel_total : 29599 time to create 1 rle with old method : 0.03384208679199219 time for calcul the mask position with numpy : 0.03746962547302246 nb_pixel_total : 24665 time to create 1 rle with old method : 0.027859926223754883 time for calcul the mask position with numpy : 0.037574052810668945 nb_pixel_total : 87877 time to create 1 rle with old method : 0.09980988502502441 time for calcul the mask position with numpy : 0.04250764846801758 nb_pixel_total : 292375 time to create 1 rle with new method : 0.6782181262969971 time for calcul the mask position with numpy : 0.03410911560058594 nb_pixel_total : 296724 time to create 1 rle with new method : 0.4736473560333252 time for calcul the mask position with numpy : 0.03958415985107422 nb_pixel_total : 288845 time to create 1 rle with new method : 0.5335719585418701 time for calcul the mask position with numpy : 0.03395199775695801 nb_pixel_total : 406913 time to create 1 rle with new method : 0.6588547229766846 time for calcul the mask position with numpy : 0.03745579719543457 nb_pixel_total : 326815 time to create 1 rle with new method : 0.49554872512817383 time for calcul the mask position with numpy : 0.02811598777770996 nb_pixel_total : 665417 time to create 1 rle with new method : 0.49692678451538086 create new chi : 4.95212197303772 time to delete rle : 0.025720834732055664 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 16737 TO DO : save crop sub photo not yet done ! save time : 2.53885817527771 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 4.773656606674194 time for calcul the mask position with numpy : 0.26299214363098145 nb_pixel_total : 5526096 time to create 1 rle with new method : 0.5389559268951416 time for calcul the mask position with numpy : 0.01977396011352539 nb_pixel_total : 66487 time to create 1 rle with old method : 0.07048892974853516 time for calcul the mask position with numpy : 0.019385099411010742 nb_pixel_total : 70924 time to create 1 rle with old method : 0.07512640953063965 time for calcul the mask position with numpy : 0.020606517791748047 nb_pixel_total : 67765 time to create 1 rle with old method : 0.07192540168762207 time for calcul the mask position with numpy : 0.023702144622802734 nb_pixel_total : 513597 time to create 1 rle with new method : 0.5195357799530029 time for calcul the mask position with numpy : 0.02210068702697754 nb_pixel_total : 15620 time to create 1 rle with old method : 0.017926692962646484 time for calcul the mask position with numpy : 0.022861719131469727 nb_pixel_total : 156180 time to create 1 rle with new method : 0.41819000244140625 time for calcul the mask position with numpy : 0.021562814712524414 nb_pixel_total : 71482 time to create 1 rle with old method : 0.1068720817565918 time for calcul the mask position with numpy : 0.02305126190185547 nb_pixel_total : 7401 time to create 1 rle with old method : 0.008263587951660156 time for calcul the mask position with numpy : 0.03432130813598633 nb_pixel_total : 337180 time to create 1 rle with new method : 0.8597683906555176 time for calcul the mask position with numpy : 0.03441143035888672 nb_pixel_total : 217508 time to create 1 rle with new method : 0.4453005790710449 create new chi : 3.7671215534210205 time to delete rle : 0.0015320777893066406 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 12564 TO DO : save crop sub photo not yet done ! save time : 2.309603691101074 nb_obj : 18 nb_hashtags : 4 time to prepare the origin masks : 6.1166770458221436 time for calcul the mask position with numpy : 0.5096828937530518 nb_pixel_total : 6676889 time to create 1 rle with new method : 0.48351216316223145 time for calcul the mask position with numpy : 0.019355297088623047 nb_pixel_total : 9929 time to create 1 rle with old method : 0.010380268096923828 time for calcul the mask position with numpy : 0.019173383712768555 nb_pixel_total : 30212 time to create 1 rle with old method : 0.031891584396362305 time for calcul the mask position with numpy : 0.019572973251342773 nb_pixel_total : 31885 time to create 1 rle with old method : 0.034261226654052734 time for calcul the mask position with numpy : 0.02010631561279297 nb_pixel_total : 6214 time to create 1 rle with old method : 0.006555318832397461 time for calcul the mask position with numpy : 0.019693613052368164 nb_pixel_total : 11203 time to create 1 rle with old method : 0.012336492538452148 time for calcul the mask position with numpy : 0.021172523498535156 nb_pixel_total : 1016 time to create 1 rle with old method : 0.0013310909271240234 time for calcul the mask position with numpy : 0.02070760726928711 nb_pixel_total : 9056 time to create 1 rle with old method : 0.009667158126831055 time for calcul the mask position with numpy : 0.02048492431640625 nb_pixel_total : 58328 time to create 1 rle with old method : 0.0628662109375 time for calcul the mask position with numpy : 0.019970178604125977 nb_pixel_total : 12084 time to create 1 rle with old method : 0.013195276260375977 time for calcul the mask position with numpy : 0.020383596420288086 nb_pixel_total : 15889 time to create 1 rle with old method : 0.01737499237060547 time for calcul the mask position with numpy : 0.020411252975463867 nb_pixel_total : 29425 time to create 1 rle with old method : 0.03161954879760742 time for calcul the mask position with numpy : 0.02022552490234375 nb_pixel_total : 10433 time to create 1 rle with old method : 0.01118016242980957 time for calcul the mask position with numpy : 0.019884347915649414 nb_pixel_total : 17005 time to create 1 rle with old method : 0.018229961395263672 time for calcul the mask position with numpy : 0.01927947998046875 nb_pixel_total : 27611 time to create 1 rle with old method : 0.030259370803833008 time for calcul the mask position with numpy : 0.020384788513183594 nb_pixel_total : 43670 time to create 1 rle with old method : 0.04721498489379883 time for calcul the mask position with numpy : 0.02016448974609375 nb_pixel_total : 10170 time to create 1 rle with old method : 0.011015653610229492 time for calcul the mask position with numpy : 0.020643949508666992 nb_pixel_total : 14323 time to create 1 rle with old method : 0.015573501586914062 time for calcul the mask position with numpy : 0.020757436752319336 nb_pixel_total : 34898 time to create 1 rle with old method : 0.03716850280761719 create new chi : 1.7950053215026855 time to delete rle : 0.0012204647064208984 batch 1 Loaded 37 chid ids of type : 3594 ++++++++++++++++++++Number RLEs to save : 8458 TO DO : save crop sub photo not yet done ! save time : 1.750945806503296 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 4.0104334354400635 time for calcul the mask position with numpy : 0.49973440170288086 nb_pixel_total : 6773240 time to create 1 rle with new method : 0.6679027080535889 time for calcul the mask position with numpy : 0.026885509490966797 nb_pixel_total : 60652 time to create 1 rle with old method : 0.0660555362701416 time for calcul the mask position with numpy : 0.020206928253173828 nb_pixel_total : 57894 time to create 1 rle with old method : 0.06217360496520996 time for calcul the mask position with numpy : 0.021303176879882812 nb_pixel_total : 46112 time to create 1 rle with old method : 0.05018782615661621 time for calcul the mask position with numpy : 0.019695520401000977 nb_pixel_total : 10556 time to create 1 rle with old method : 0.011444330215454102 time for calcul the mask position with numpy : 0.026252269744873047 nb_pixel_total : 13504 time to create 1 rle with old method : 0.015525579452514648 time for calcul the mask position with numpy : 0.023552417755126953 nb_pixel_total : 5973 time to create 1 rle with old method : 0.00689697265625 time for calcul the mask position with numpy : 0.02311849594116211 nb_pixel_total : 14260 time to create 1 rle with old method : 0.0164339542388916 time for calcul the mask position with numpy : 0.02275228500366211 nb_pixel_total : 8922 time to create 1 rle with old method : 0.010428905487060547 time for calcul the mask position with numpy : 0.02235579490661621 nb_pixel_total : 16220 time to create 1 rle with old method : 0.018950462341308594 time for calcul the mask position with numpy : 0.022193193435668945 nb_pixel_total : 19251 time to create 1 rle with old method : 0.022297143936157227 time for calcul the mask position with numpy : 0.022762775421142578 nb_pixel_total : 23656 time to create 1 rle with old method : 0.02643895149230957 create new chi : 1.7711856365203857 time to delete rle : 0.0008871555328369141 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 6042 TO DO : save crop sub photo not yet done ! save time : 1.2691819667816162 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 3.6969094276428223 time for calcul the mask position with numpy : 0.5384790897369385 nb_pixel_total : 6658096 time to create 1 rle with new method : 0.5475037097930908 time for calcul the mask position with numpy : 0.03132915496826172 nb_pixel_total : 49901 time to create 1 rle with old method : 0.05227375030517578 time for calcul the mask position with numpy : 0.019658327102661133 nb_pixel_total : 9303 time to create 1 rle with old method : 0.009745597839355469 time for calcul the mask position with numpy : 0.021810293197631836 nb_pixel_total : 236020 time to create 1 rle with new method : 0.8385887145996094 time for calcul the mask position with numpy : 0.02032303810119629 nb_pixel_total : 8131 time to create 1 rle with old method : 0.008593320846557617 time for calcul the mask position with numpy : 0.01988840103149414 nb_pixel_total : 18461 time to create 1 rle with old method : 0.01952672004699707 time for calcul the mask position with numpy : 0.020217180252075195 nb_pixel_total : 40591 time to create 1 rle with old method : 0.042784690856933594 time for calcul the mask position with numpy : 0.02026224136352539 nb_pixel_total : 23546 time to create 1 rle with old method : 0.025330066680908203 time for calcul the mask position with numpy : 0.019524097442626953 nb_pixel_total : 6191 time to create 1 rle with old method : 0.006604671478271484 create new chi : 2.3221943378448486 time to delete rle : 0.0009031295776367188 batch 1 Loaded 17 chid ids of type : 3594 +++++++++Number RLEs to save : 6544 TO DO : save crop sub photo not yet done ! save time : 0.8927130699157715 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 2.6241655349731445 time for calcul the mask position with numpy : 1.4399628639221191 nb_pixel_total : 6563032 time to create 1 rle with new method : 2.104840040206909 time for calcul the mask position with numpy : 0.02071547508239746 nb_pixel_total : 43723 time to create 1 rle with old method : 0.047048091888427734 time for calcul the mask position with numpy : 0.019710302352905273 nb_pixel_total : 51019 time to create 1 rle with old method : 0.05391502380371094 time for calcul the mask position with numpy : 0.019588947296142578 nb_pixel_total : 89400 time to create 1 rle with old method : 0.09484457969665527 time for calcul the mask position with numpy : 0.019681692123413086 nb_pixel_total : 42322 time to create 1 rle with old method : 0.04555869102478027 time for calcul the mask position with numpy : 0.020192384719848633 nb_pixel_total : 88534 time to create 1 rle with old method : 0.09263730049133301 time for calcul the mask position with numpy : 0.020364999771118164 nb_pixel_total : 145225 time to create 1 rle with old method : 0.1508345603942871 time for calcul the mask position with numpy : 0.0196225643157959 nb_pixel_total : 14276 time to create 1 rle with old method : 0.014930009841918945 time for calcul the mask position with numpy : 0.019284963607788086 nb_pixel_total : 12709 time to create 1 rle with old method : 0.013168096542358398 create new chi : 4.256036996841431 time to delete rle : 0.0009565353393554688 batch 1 Loaded 17 chid ids of type : 3594 +++++++++++Number RLEs to save : 6794 TO DO : save crop sub photo not yet done ! save time : 1.902393102645874 nb_obj : 12 nb_hashtags : 2 time to prepare the origin masks : 4.8399598598480225 time for calcul the mask position with numpy : 0.42632508277893066 nb_pixel_total : 6249139 time to create 1 rle with new method : 0.6546247005462646 time for calcul the mask position with numpy : 0.019880294799804688 nb_pixel_total : 6659 time to create 1 rle with old method : 0.007043600082397461 time for calcul the mask position with numpy : 0.019303560256958008 nb_pixel_total : 24312 time to create 1 rle with old method : 0.025678396224975586 time for calcul the mask position with numpy : 0.019245147705078125 nb_pixel_total : 18035 time to create 1 rle with old method : 0.020256757736206055 time for calcul the mask position with numpy : 0.019493818283081055 nb_pixel_total : 62430 time to create 1 rle with old method : 0.06772828102111816 time for calcul the mask position with numpy : 0.019889116287231445 nb_pixel_total : 138324 time to create 1 rle with old method : 0.1461653709411621 time for calcul the mask position with numpy : 0.019383668899536133 nb_pixel_total : 18728 time to create 1 rle with old method : 0.019582271575927734 time for calcul the mask position with numpy : 0.02047133445739746 nb_pixel_total : 19801 time to create 1 rle with old method : 0.021519184112548828 time for calcul the mask position with numpy : 0.019670963287353516 nb_pixel_total : 15372 time to create 1 rle with old method : 0.017197847366333008 time for calcul the mask position with numpy : 0.019782304763793945 nb_pixel_total : 36961 time to create 1 rle with old method : 0.040465354919433594 time for calcul the mask position with numpy : 0.021091699600219727 nb_pixel_total : 151838 time to create 1 rle with new method : 0.43802642822265625 time for calcul the mask position with numpy : 0.024726152420043945 nb_pixel_total : 222449 time to create 1 rle with new method : 2.6692535877227783 time for calcul the mask position with numpy : 0.02082538604736328 nb_pixel_total : 86192 time to create 1 rle with old method : 0.09003853797912598 create new chi : 4.975416421890259 time to delete rle : 0.0014765262603759766 batch 1 Loaded 25 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 10174 TO DO : save crop sub photo not yet done ! save time : 2.223188638687134 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 3.665620803833008 time for calcul the mask position with numpy : 0.6176435947418213 nb_pixel_total : 6468428 time to create 1 rle with new method : 1.5187137126922607 time for calcul the mask position with numpy : 0.01927924156188965 nb_pixel_total : 6330 time to create 1 rle with old method : 0.006547689437866211 time for calcul the mask position with numpy : 0.019834280014038086 nb_pixel_total : 13955 time to create 1 rle with old method : 0.014656543731689453 time for calcul the mask position with numpy : 0.019000768661499023 nb_pixel_total : 41919 time to create 1 rle with old method : 0.04344296455383301 time for calcul the mask position with numpy : 0.021489620208740234 nb_pixel_total : 293961 time to create 1 rle with new method : 0.6861720085144043 time for calcul the mask position with numpy : 0.020902156829833984 nb_pixel_total : 24161 time to create 1 rle with old method : 0.02720046043395996 time for calcul the mask position with numpy : 0.02035975456237793 nb_pixel_total : 45263 time to create 1 rle with old method : 0.04947686195373535 time for calcul the mask position with numpy : 0.0205233097076416 nb_pixel_total : 67554 time to create 1 rle with old method : 0.07181930541992188 time for calcul the mask position with numpy : 0.02047872543334961 nb_pixel_total : 69955 time to create 1 rle with old method : 0.07600760459899902 time for calcul the mask position with numpy : 0.02023625373840332 nb_pixel_total : 18714 time to create 1 rle with old method : 0.019823551177978516 create new chi : 3.3706579208374023 time to delete rle : 0.0011913776397705078 batch 1 Loaded 19 chid ids of type : 3594 +++++++++++++Number RLEs to save : 7803 TO DO : save crop sub photo not yet done ! save time : 0.9627220630645752 map_output_result : {1360648563: (0.0, 'Should be the crop_list due to order', 0), 1360648521: (0.0, 'Should be the crop_list due to order', 0), 1360623490: (0.0, 'Should be the crop_list due to order', 0), 1360623484: (0.0, 'Should be the crop_list due to order', 0), 1360623480: (0.0, 'Should be the crop_list due to order', 0), 1360623466: (0.0, 'Should be the crop_list due to order', 0), 1360623387: (0.0, 'Should be the crop_list due to order', 0), 1360621971: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1360648563, 1360648521, 1360623490, 1360623484, 1360623480, 1360623466, 1360623387, 1360621971] Looping around the photos to save general results len do output : 8 /1360648563.Didn't retrieve data . /1360648521.Didn't retrieve data . /1360623490.Didn't retrieve data . /1360623484.Didn't retrieve data . /1360623480.Didn't retrieve data . /1360623466.Didn't retrieve data . /1360623387.Didn't retrieve data . /1360621971.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648563', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648521', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623490', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623484', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623480', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623466', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623387', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360621971', None, None, None, None, None, '2961022') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.016193389892578125 save_final save missing photos in datou_result : time spend for datou_step_exec : 75.69001698493958 time spend to save output : 0.016568660736083984 total time spend for step 3 : 75.70658564567566 step4:ventilate_hashtags_in_portfolio Tue May 27 01:33:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23360449 get user id for portfolio 23360449 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23360449 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','background','pehd','pet_clair','mal_croppe','papier','carton','environnement','flou','pet_fonce','autre')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23360449 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','background','pehd','pet_clair','mal_croppe','papier','carton','environnement','flou','pet_fonce','autre')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23360449 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','background','pehd','pet_clair','mal_croppe','papier','carton','environnement','flou','pet_fonce','autre')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23361841,23361842,23361843,23361844,23361845,23361846,23361847,23361848,23361849,23361850,23361851?tags=metal,background,pehd,pet_clair,mal_croppe,papier,carton,environnement,flou,pet_fonce,autre Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1360648563, 1360648521, 1360623490, 1360623484, 1360623480, 1360623466, 1360623387, 1360621971] Looping around the photos to save general results len do output : 1 /23360449. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648563', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648521', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623490', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623484', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623480', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623466', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623387', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360621971', None, None, None, None, None, '2961022') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.015591144561767578 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.558197021484375 time spend to save output : 0.01589179039001465 total time spend for step 4 : 1.5740888118743896 step5:final Tue May 27 01:33:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1360648563: ('0.12451613349332791',), 1360648521: ('0.12451613349332791',), 1360623490: ('0.12451613349332791',), 1360623484: ('0.12451613349332791',), 1360623480: ('0.12451613349332791',), 1360623466: ('0.12451613349332791',), 1360623387: ('0.12451613349332791',), 1360621971: ('0.12451613349332791',)} new output for save of step final : {1360648563: ('0.12451613349332791',), 1360648521: ('0.12451613349332791',), 1360623490: ('0.12451613349332791',), 1360623484: ('0.12451613349332791',), 1360623480: ('0.12451613349332791',), 1360623466: ('0.12451613349332791',), 1360623387: ('0.12451613349332791',), 1360621971: ('0.12451613349332791',)} [1360648563, 1360648521, 1360623490, 1360623484, 1360623480, 1360623466, 1360623387, 1360621971] Looping around the photos to save general results len do output : 8 /1360648563.Didn't retrieve data . /1360648521.Didn't retrieve data . /1360623490.Didn't retrieve data . /1360623484.Didn't retrieve data . /1360623480.Didn't retrieve data . /1360623466.Didn't retrieve data . /1360623387.Didn't retrieve data . /1360621971.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648563', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648521', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623490', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623484', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623480', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623466', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623387', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360621971', None, None, None, None, None, '2961022') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.015294313430786133 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10415387153625488 time spend to save output : 0.015760421752929688 total time spend for step 5 : 0.11991429328918457 step6:blur_detection Tue May 27 01:33:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd.jpg resize: (2160, 3264) 1360648563 0.700039562711206 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be.jpg resize: (2160, 3264) 1360648521 1.8502824601779173 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8.jpg resize: (2160, 3264) 1360623490 -0.5069968802584591 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b.jpg resize: (2160, 3264) 1360623484 -2.0750246210366092 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a.jpg resize: (2160, 3264) 1360623480 -1.1236577398137564 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391.jpg resize: (2160, 3264) 1360623466 2.7319755530238794 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784.jpg resize: (2160, 3264) 1360623387 4.271685162558746 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493.jpg resize: (2160, 3264) 1360621971 -0.3475912080903197 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627692_0.png resize: (1063, 772) 1361191094 -0.3399438212905859 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627695_0.png resize: (939, 524) 1361191096 -1.132243620747654 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627696_0.png resize: (694, 700) 1361191098 0.7727617093138358 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627698_0.png resize: (252, 217) 1361191099 -1.7221520201138973 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627699_0.png resize: (450, 110) 1361191102 -2.4991511613656705 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627700_0.png resize: (208, 353) 1361191103 -1.6000919166930094 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627701_0.png resize: (284, 250) 1361191106 -1.1580156598579272 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627703_0.png resize: (104, 228) 1361191108 -2.3134034761422844 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627704_0.png resize: (179, 230) 1361191109 -1.3913879663940079 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627705_0.png resize: (776, 595) 1361191111 -1.5150357933208596 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627706_0.png resize: (834, 1008) 1361191113 -0.5356227084038025 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627707_0.png resize: (98, 98) 1361191114 0.3291257609234272 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627708_0.png resize: (273, 420) 1361191116 -0.9432943569734079 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627709_0.png resize: (595, 362) 1361191118 -1.174260850325376 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627710_0.png resize: (208, 112) 1361191119 -2.055527189247404 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627711_0.png resize: (688, 1079) 1361191120 0.021479361033135438 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627714_0.png resize: (219, 475) 1361191122 -0.36909157444639745 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627715_0.png resize: (179, 273) 1361191124 -1.4747247680784195 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627716_0.png resize: (177, 117) 1361191125 -0.2850052182489365 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627717_0.png resize: (114, 128) 1361191127 -0.4846886574328236 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627718_0.png resize: (335, 354) 1361191129 0.2616326941880826 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627720_0.png resize: (151, 146) 1361191130 -0.3581136593459383 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627722_0.png resize: (266, 218) 1361191131 -0.5592503586961538 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627723_0.png resize: (171, 207) 1361191133 -1.1731748300512739 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627724_0.png resize: (144, 145) 1361191135 -1.5882052201013606 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627726_0.png resize: (129, 90) 1361191136 -0.49731033310733574 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627729_0.png resize: (173, 93) 1361191138 -0.12549926670921135 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627732_0.png resize: (186, 74) 1361191140 -1.062107316714639 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627733_0.png resize: (179, 199) 1361191141 -0.6488750337313649 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627735_0.png resize: (142, 167) 1361191143 -1.250339933958247 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627736_0.png resize: (148, 74) 1361191145 0.009214394835002804 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627737_0.png resize: (139, 151) 1361191146 0.20262329321268202 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627739_0.png resize: (124, 132) 1361191149 0.11087663939921896 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627740_0.png resize: (159, 118) 1361191150 -2.7834765323582973 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627744_0.png resize: (122, 86) 1361191153 -2.248573167909466 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627745_0.png resize: (214, 172) 1361191154 1.752244917410215 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627747_0.png resize: (164, 142) 1361191156 -0.7740522646561673 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627748_0.png resize: (115, 94) 1361191157 0.19353043102096046 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627749_0.png resize: (870, 357) 1361191159 -1.6842477702294691 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627752_0.png resize: (143, 142) 1361191161 -1.495946721235179 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627753_0.png resize: (108, 173) 1361191163 -2.2036614065717712 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627754_0.png resize: (434, 425) 1361191164 -1.1295274999000051 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627756_0.png resize: (266, 255) 1361191166 -1.667082416220098 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627757_0.png resize: (236, 649) 1361191167 -0.7707751226633348 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627758_0.png resize: (419, 173) 1361191168 -1.565366934781514 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627760_0.png resize: (399, 348) 1361191170 0.9687682080028767 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627761_0.png resize: (418, 656) 1361191171 1.2131135403742992 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627763_0.png resize: (338, 167) 1361191172 1.7494519243280031 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627764_0.png resize: (186, 148) 1361191174 -1.7354775988067634 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627766_0.png resize: (229, 114) 1361191175 -0.311499831750587 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627767_0.png resize: (417, 591) 1361191176 1.5611009173840882 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627768_0.png resize: (335, 243) 1361191177 1.0742804193567428 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627772_0.png resize: (142, 201) 1361191179 -1.2576940717102625 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627773_0.png resize: (339, 321) 1361191180 -1.2346295425403477 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627774_0.png resize: (275, 401) 1361191181 3.332453518571506 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627775_0.png resize: (341, 255) 1361191183 -0.2680088101650864 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627776_0.png resize: (263, 183) 1361191184 -0.8788017159022782 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627778_0.png resize: (281, 262) 1361191185 -0.3779635449566716 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627780_0.png resize: (121, 79) 1361191186 -0.43001479817064464 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627691_0.png resize: (740, 1183) 1361191233 -0.12725890568234458 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627712_0.png resize: (487, 301) 1361191234 -0.8379031292140245 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627719_0.png resize: (180, 219) 1361191235 -0.31071835332815084 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627727_0.png resize: (148, 153) 1361191237 -1.6263998497371805 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627730_0.png resize: (209, 240) 1361191238 -1.5793327374679056 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627741_0.png resize: (207, 344) 1361191239 -1.7464844678620377 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627750_0.png resize: (109, 135) 1361191241 -0.6885661937679826 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627759_0.png resize: (301, 193) 1361191242 -1.2168393764357264 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627694_0.png resize: (495, 858) 1361191357 0.4776234763627948 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627697_0.png resize: (374, 322) 1361191358 -2.635059761631367 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627702_0.png resize: (138, 359) 1361191361 -1.4262919066035498 treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627713_0.png resize: (302, 333) 1361191363 -0.11741089487878213 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627725_0.png resize: (244, 297) 1361191364 -0.896836585523457 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627728_0.png resize: (75, 202) 1361191366 -2.815179886130889 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627731_0.png resize: (168, 231) 1361191367 -3.3137782620536482 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627734_0.png resize: (126, 186) 1361191369 -1.4027339957509595 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627738_0.png resize: (94, 95) 1361191370 -1.7368200938646488 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627742_0.png resize: (254, 273) 1361191372 -0.8300681696070303 treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627743_0.png resize: (237, 363) 1361191374 -1.375608291100969 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627746_0.png resize: (190, 268) 1361191375 -0.5964369929634307 treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627751_0.png resize: (283, 238) 1361191377 -2.19500726307387 treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627755_0.png resize: (296, 395) 1361191378 -1.401729721302888 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627762_0.png resize: (528, 555) 1361191380 0.23104376623289316 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627765_0.png resize: (188, 161) 1361191381 0.2601511309313573 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627769_0.png resize: (346, 281) 1361191383 1.575542501272469 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627770_0.png resize: (238, 158) 1361191385 0.4273753466945971 treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627771_0.png resize: (71, 124) 1361191386 1.0920752935201077 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627777_0.png resize: (716, 593) 1361191388 -1.9464619896423498 treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627779_0.png resize: (132, 135) 1361191390 -3.5631865144084474 treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627693_0.png resize: (729, 777) 1361191458 -0.37168187799809976 treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627721_0.png resize: (93, 133) 1361191497 2.6125997062305357 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 98 time used for this insertion : 0.01737046241760254 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 98 time used for this insertion : 0.027021408081054688 save missing photos in datou_result : time spend for datou_step_exec : 26.56225037574768 time spend to save output : 0.04948568344116211 total time spend for step 6 : 26.611736059188843 step7:brightness Tue May 27 01:34:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd.jpg treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be.jpg treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8.jpg treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b.jpg treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a.jpg treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391.jpg treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784.jpg treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493.jpg treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627692_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627695_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627696_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627698_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627699_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627700_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627701_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627703_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627704_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627705_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627706_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627707_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627708_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627709_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627710_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627711_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627714_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627715_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627716_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627717_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627718_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627720_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627722_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627723_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627724_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627726_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627729_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627732_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627733_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627735_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627736_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627737_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627739_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627740_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627744_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627745_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627747_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627748_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627749_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627752_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627753_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627754_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627756_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627757_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627758_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627760_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627761_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627763_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627764_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627766_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627767_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627768_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627772_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627773_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627774_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627775_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627776_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627778_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627780_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627691_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627712_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627719_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627727_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627730_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627741_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627750_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627759_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627694_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627697_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627702_0.png treat image : temp/1748302228_2365930_1360648521_1e08ddd70180e9b786899ecf28ac95be_rle_crop_3813627713_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627725_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627728_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627731_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627734_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627738_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627742_0.png treat image : temp/1748302228_2365930_1360623484_f3e8b23b858328d5f34693905f28304b_rle_crop_3813627743_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627746_0.png treat image : temp/1748302228_2365930_1360623480_67e9f631ccaa58a37b03850d9338e61a_rle_crop_3813627751_0.png treat image : temp/1748302228_2365930_1360623466_b5974db955b8769ec085d8ec82cf6391_rle_crop_3813627755_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627762_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627765_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627769_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627770_0.png treat image : temp/1748302228_2365930_1360623387_291f152469968c811bb9d5c6d737a784_rle_crop_3813627771_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627777_0.png treat image : temp/1748302228_2365930_1360621971_6de1633e49b0e625284a2514bd172493_rle_crop_3813627779_0.png treat image : temp/1748302228_2365930_1360648563_b9d83f33f94cdf520454b3f7f65c93dd_rle_crop_3813627693_0.png treat image : temp/1748302228_2365930_1360623490_153dc2ec9d994677673663bd3fd5ffe8_rle_crop_3813627721_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 98 time used for this insertion : 0.023894786834716797 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 98 time used for this insertion : 0.025795459747314453 save missing photos in datou_result : time spend for datou_step_exec : 7.132046937942505 time spend to save output : 0.05476832389831543 total time spend for step 7 : 7.18681526184082 step8:velours_tree Tue May 27 01:34:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.09044861793518066 time spend to save output : 4.00543212890625e-05 total time spend for step 8 : 0.09048867225646973 step9:send_mail_cod Tue May 27 01:34:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23360449_27-05-2025_01_34_13.pdf 23361841 imagette233618411748302453 23361842 imagette233618421748302453 23361843 change filename to text .imagette233618431748302453 23361844 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette233618441748302453 23361845 imagette233618451748302454 23361846 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette233618461748302454 23361847 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette233618471748302456 23361849 imagette233618491748302456 23361850 imagette233618501748302456 23361851 change filename to text .imagette233618511748302456 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23360449 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23361841,23361842,23361843,23361844,23361845,23361846,23361847,23361848,23361849,23361850,23361851?tags=metal,background,pehd,pet_clair,mal_croppe,papier,carton,environnement,flou,pet_fonce,autre args[1360648563] : ((1360648563, 0.700039562711206, 492688767), (1360648563, 0.031232187149077932, 2107752395), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360648521] : ((1360648521, 1.8502824601779173, 492688767), (1360648521, 0.04158666138410118, 2107752395), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360623490] : ((1360623490, -0.5069968802584591, 492688767), (1360623490, 0.16238827279012244, 2107752395), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360623484] : ((1360623484, -2.0750246210366092, 492609224), (1360623484, 0.16572429195931296, 2107752395), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360623480] : ((1360623480, -1.1236577398137564, 492688767), (1360623480, -0.036360263533665706, 2107752395), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360623466] : ((1360623466, 2.7319755530238794, 492688767), (1360623466, -0.1962235731072406, 496442774), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360623387] : ((1360623387, 4.271685162558746, 492688767), (1360623387, -0.08215687792292088, 496442774), '0.12451613349332791') We are sending mail with results at report@fotonower.com args[1360621971] : ((1360621971, -0.3475912080903197, 492688767), (1360621971, 0.33728408729689263, 2107752395), '0.12451613349332791') We are sending mail with results at report@fotonower.com refus_total : 0.12451613349332791 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23360449 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23360449_27-05-2025_01_34_13.pdf results_Auto_P23360449_27-05-2025_01_34_13.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23360449_27-05-2025_01_34_13.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23360449','results_Auto_P23360449_27-05-2025_01_34_13.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23360449_27-05-2025_01_34_13.pdf','pdf','','0.59','0.12451613349332791') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23360449

https://www.fotonower.com/image?json=false&list_photos_id=1360648563
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360648521
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.8502824601779173)
https://www.fotonower.com/image?json=false&list_photos_id=1360623490
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360623484
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360623480
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360623466
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.7319755530238794)
https://www.fotonower.com/image?json=false&list_photos_id=1360623387
La photo est trop floue, merci de reprendre une photo.(avec le score = 4.271685162558746)
https://www.fotonower.com/image?json=false&list_photos_id=1360621971
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 12.45%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pehd: https://www.fotonower.com/view/23361843?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23361844?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23361846?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23361847?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23361851?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23360449_27-05-2025_01_34_13.pdf.

Lien vers velours :https://www.fotonower.com/velours/23361841,23361842,23361843,23361844,23361845,23361846,23361847,23361848,23361849,23361850,23361851?tags=metal,background,pehd,pet_clair,mal_croppe,papier,carton,environnement,flou,pet_fonce,autre.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 26 May 2025 23:34:19 GMT Content-Length: 0 Connection: close X-Message-Id: uSdQBx9zTcSp9mFqz5j5tg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1360648563, 1360648521, 1360623490, 1360623484, 1360623480, 1360623466, 1360623387, 1360621971] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648563', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648521', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623490', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623484', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623480', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623466', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623387', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360621971', None, None, None, None, None, '2961022') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.04375863075256348 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.7882421016693115 time spend to save output : 0.0439760684967041 total time spend for step 9 : 5.832218170166016 step10:split_time_score Tue May 27 01:34:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 22052025 23360449 Nombre de photos uploadées : 8 / 23040 (0%) 22052025 23360449 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 22052025 23360449 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.00040340423583984375 elapsed_time : insert_dashboard_record_day_entry 0.03643631935119629 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.12898913724922847 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23263226_22-05-2025_11_15_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23263226 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23263226 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360420 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23267925 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23265122 order by id desc limit 1 Qualite : 0.1383823290524787 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23269440_26-05-2025_11_33_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23269440 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23269440 AND mptpi.`type`=3594 To do Qualite : 0.156928577372307 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23267928_22-05-2025_13_48_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23267928 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23267928 AND mptpi.`type`=3594 To do Qualite : 0.13890153030251456 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23267932_22-05-2025_13_34_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23267932 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23267932 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360434 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360436 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360438 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360445 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360447 order by id desc limit 1 Qualite : 0.12451613349332791 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23360449_27-05-2025_01_34_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23360449 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23360449 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'22052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1360648563, 1360648521, 1360623490, 1360623484, 1360623480, 1360623466, 1360623387, 1360621971] Looping around the photos to save general results len do output : 1 /23360449Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648563', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360648521', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623490', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623484', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623480', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623466', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360623387', None, None, None, None, None, '2961022') ('3318', None, None, None, None, None, None, None, '2961022') ('3318', '23360449', '1360621971', None, None, None, None, None, '2961022') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.02282238006591797 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.7231364250183105 time spend to save output : 0.023037433624267578 total time spend for step 10 : 7.746173858642578 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 101.85user 80.39system 4:02.41elapsed 75%CPU (0avgtext+0avgdata 4106316maxresident)k 722712inputs+62656outputs (1477major+8737563minor)pagefaults 0swaps