python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2135421 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2977366'] with mtr_portfolio_ids : ['23420184'] and first list_photo_ids : [] new path : /proc/2135421/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 1.3367748260498047 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed May 28 16:50:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-28 16:50:34.755734: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-28 16:50:34.783134: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-28 16:50:34.784725: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2270000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-28 16:50:34.784776: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-28 16:50:34.787259: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-28 16:50:35.099764: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2b290b70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-28 16:50:35.099824: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-28 16:50:35.100838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-28 16:50:35.101195: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-28 16:50:35.103464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-28 16:50:35.105816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-28 16:50:35.106159: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-28 16:50:35.108604: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-28 16:50:35.109689: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-28 16:50:35.114463: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-28 16:50:35.116017: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-28 16:50:35.116104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-28 16:50:35.116878: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-28 16:50:35.116894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-28 16:50:35.116904: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-28 16:50:35.118243: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-28 16:50:35.386917: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-28 16:50:35.387013: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-28 16:50:35.387035: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-28 16:50:35.387054: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-28 16:50:35.387071: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-28 16:50:35.387089: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-28 16:50:35.387107: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-28 16:50:35.387125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-28 16:50:35.388742: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-28 16:50:35.390156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-28 16:50:35.390209: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-28 16:50:35.390228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-28 16:50:35.390246: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-28 16:50:35.390263: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-28 16:50:35.390280: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-28 16:50:35.390297: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-28 16:50:35.390315: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-28 16:50:35.391945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-28 16:50:35.391987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-28 16:50:35.391999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-28 16:50:35.392009: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-28 16:50:35.393462: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-28 16:50:46.534918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-28 16:50:46.720482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 15.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 10.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 Detection mask done ! Trying to reset tf kernel 2135877 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0011811256408691406 nb_pixel_total : 43331 time to create 1 rle with old method : 0.05037355422973633 length of segment : 172 time for calcul the mask position with numpy : 0.0009088516235351562 nb_pixel_total : 35825 time to create 1 rle with old method : 0.04146242141723633 length of segment : 257 time for calcul the mask position with numpy : 0.00033783912658691406 nb_pixel_total : 10032 time to create 1 rle with old method : 0.011809825897216797 length of segment : 94 time for calcul the mask position with numpy : 0.0007920265197753906 nb_pixel_total : 36525 time to create 1 rle with old method : 0.041381120681762695 length of segment : 227 time for calcul the mask position with numpy : 0.0001659393310546875 nb_pixel_total : 7622 time to create 1 rle with old method : 0.008987665176391602 length of segment : 82 time for calcul the mask position with numpy : 0.0010221004486083984 nb_pixel_total : 52837 time to create 1 rle with old method : 0.059270620346069336 length of segment : 274 time for calcul the mask position with numpy : 0.047048091888427734 nb_pixel_total : 1150820 time to create 1 rle with new method : 0.09545350074768066 length of segment : 1059 time for calcul the mask position with numpy : 0.0006804466247558594 nb_pixel_total : 18418 time to create 1 rle with old method : 0.03224444389343262 length of segment : 168 time for calcul the mask position with numpy : 0.00028824806213378906 nb_pixel_total : 12711 time to create 1 rle with old method : 0.014168024063110352 length of segment : 141 time for calcul the mask position with numpy : 0.0007646083831787109 nb_pixel_total : 41794 time to create 1 rle with old method : 0.0450589656829834 length of segment : 199 time for calcul the mask position with numpy : 0.001512289047241211 nb_pixel_total : 70968 time to create 1 rle with old method : 0.07651257514953613 length of segment : 418 time for calcul the mask position with numpy : 0.0002512931823730469 nb_pixel_total : 9174 time to create 1 rle with old method : 0.010246992111206055 length of segment : 111 time for calcul the mask position with numpy : 0.0035212039947509766 nb_pixel_total : 168100 time to create 1 rle with new method : 0.013080596923828125 length of segment : 622 time for calcul the mask position with numpy : 0.0008945465087890625 nb_pixel_total : 40274 time to create 1 rle with old method : 0.04969453811645508 length of segment : 262 time for calcul the mask position with numpy : 0.0044307708740234375 nb_pixel_24256 time to create 1 rle with old method : 0.027485370635986328 length of segment : 206 time for calcul the mask position with numpy : 0.0012633800506591797 nb_pixel_total : 58854 time to create 1 rle with old method : 0.07361102104187012 length of segment : 427 time for calcul the mask position with numpy : 0.00667119026184082 nb_pixel_total : 238513 time to create 1 rle with new method : 0.017189502716064453 length of segment : 730 time for calcul the mask position with numpy : 0.0013811588287353516 nb_pixel_total : 56408 time to create 1 rle with old method : 0.0650644302368164 length of segment : 297 time for calcul the mask position with numpy : 0.00044345855712890625 nb_pixel_total : 20657 time to create 1 rle with old method : 0.03389763832092285 length of segment : 141 time for calcul the mask position with numpy : 0.0005903244018554688 nb_pixel_total : 20079 time to create 1 rle with old method : 0.02881765365600586 length of segment : 247 time for calcul the mask position with numpy : 0.0003044605255126953 nb_pixel_total : 10648 time to create 1 rle with old method : 0.012335062026977539 length of segment : 137 time for calcul the mask position with numpy : 0.003745555877685547 nb_pixel_total : 180592 time to create 1 rle with new method : 0.010291576385498047 length of segment : 592 time for calcul the mask position with numpy : 0.0011074542999267578 nb_pixel_total : 46407 time to create 1 rle with old method : 0.0762934684753418 length of segment : 198 time for calcul the mask position with numpy : 0.0005221366882324219 nb_pixel_total : 22015 time to create 1 rle with old method : 0.025562524795532227 length of segment : 250 time for calcul the mask position with numpy : 0.0004980564117431641 nb_pixel_total : 14117 time to create 1 rle with old method : 0.016621828079223633 length of segment : 159 time for calcul the mask position with numpy : 0.0006823539733886719 nb_pixel_total : 33196 time to create 1 rle with old method : 0.03836488723754883 length of segment : 202 time for calcul the mask position with numpy : 0.005934476852416992 nb_pixel_total : 344032 time to create 1 rle with new method : 0.01525259017944336 length of segment : 996 time for calcul the mask position with numpy : 0.018281936645507812 nb_pixel_total : 677470 time to create 1 rle with new method : 0.7618341445922852 length of segment : 1346 time for calcul the mask position with numpy : 0.0005805492401123047 nb_pixel_total : 15727 time to create 1 rle with old method : 0.021222352981567383 length of segment : 239 time for calcul the mask position with numpy : 0.0021300315856933594 nb_pixel_total : 102720 time to create 1 rle with old method : 0.11804747581481934 length of segment : 413 time for calcul the mask position with numpy : 0.002204418182373047 nb_pixel_total : 114801 time to create 1 rle with old method : 0.12679815292358398 length of segment : 501 time for calcul the mask position with numpy : 0.00103759765625 nb_pixel_total : 18786 time to create 1 rle with old method : 0.021953582763671875 length of segment : 174 time for calcul the mask position with numpy : 0.0010101795196533203 nb_pixel_total : 17484 time to create 1 rle with old method : 0.019588232040405273 length of segment : 225 time for calcul the mask position with numpy : 0.0020945072174072266 nb_pixel_total : 40211 time to create 1 rle with old method : 0.04630923271179199 length of segment : 139 time for calcul the mask position with numpy : 0.007386207580566406 nb_pixel_total : 258225 time to create 1 rle with new method : 0.011113166809082031 length of segment : 480 time for calcul the mask position with numpy : 0.00318145751953125 nb_pixel_total : 95681 time to create 1 rle with old method : 0.10848879814147949 length of segment : 301 time for calcul the mask position with numpy : 0.002887248992919922 nb_pixel_total : 98455 time to create 1 rle with old method : 0.10976910591125488 length of segment : 296 time for calcul the mask position with numpy : 0.0011272430419921875 nb_pixel_total : 28421 time to create 1 rle with old method : 0.033095598220825195 length of segment : 179 time for calcul the mask position with numpy : 0.0006811618804931641 nb_pixel_total : 15772 time to create 1 rle with old method : 0.01867985725402832 length of segment : 150 time for calcul the mask position with numpy : 0.0006079673767089844 nb_pixel_total : 18774 time to create 1 rle with old method : 0.023734092712402344 length of segment : 179 time for calcul the mask position with numpy : 0.0004837512969970703 nb_pixel_total : 9456 time to create 1 rle with old method : 0.015976428985595703 length of segment : 132 time for calcul the mask position with numpy : 0.0012135505676269531 nb_pixel_total : 33669 time to create 1 rle with old method : 0.04172158241271973 length of segment : 269 time for calcul the mask position with numpy : 0.0014829635620117188 nb_pixel_total : 45517 time to create 1 rle with old method : 0.05822348594665527 length of segment : 267 time for calcul the mask position with numpy : 0.002214670181274414 nb_pixel_total : 45616 time to create 1 rle with old method : 0.05590367317199707 length of segment : 309 time for calcul the mask position with numpy : 0.0019011497497558594 nb_pixel_total : 66336 time to create 1 rle with old method : 0.0765848159790039 length of segment : 398 time for calcul the mask position with numpy : 0.0006175041198730469 nb_pixel_total : 13693 time to create 1 rle with old method : 0.015580415725708008 length of segment : 158 time for calcul the mask position with numpy : 0.0014085769653320312 nb_pixel_total : 25801 time to create 1 rle with old method : 0.03220367431640625 length of segment : 226 time for calcul the mask position with numpy : 0.0010232925415039062 nb_pixel_total : 18119 time to create 1 rle with old method : 0.02116537094116211 length of segment : 245 time for calcul the mask position with numpy : 0.0005180835723876953 nb_pixel_total : 11967 time to create 1 rle with old method : 0.014173269271850586 length of segment : 141 time for calcul the mask position with numpy : 0.0008442401885986328 nb_pixel_total : 16186 time to create 1 rle with old method : 0.017778635025024414 length of segment : 235 time for calcul the mask position with numpy : 0.0004286766052246094 nb_pixel_total : 13686 time to create 1 rle with old method : 0.015924930572509766 length of segment : 126 time for calcul the mask position with numpy : 0.000858306884765625 nb_pixel_total : 17564 time to create 1 rle with old method : 0.02007770538330078 length of segment : 203 time for calcul the mask position with numpy : 0.0004258155822753906 nb_pixel_total : 12153 time to create 1 rle with old method : 0.018334150314331055 length of segment : 78 time for calcul the mask position with numpy : 0.0008575916290283203 nb_pixel_total : 22182 time to create 1 rle with old method : 0.028669357299804688 length of segment : 168 time for calcul the mask position with numpy : 0.010379314422607422 nb_pixel_total : 198791 time to create 1 rle with new method : 0.015932083129882812 length of segment : 556 time for calcul the mask position with numpy : 0.0007355213165283203 nb_pixel_total : 13636 time to create 1 rle with old method : 0.015652894973754883 length of segment : 149 time for calcul the mask position with numpy : 0.0025992393493652344 nb_pixel_total : 78268 time to create 1 rle with old method : 0.08845973014831543 length of segment : 500 time for calcul the mask position with numpy : 0.0019428730010986328 nb_pixel_total : 43604 time to create 1 rle with old method : 0.05163884162902832 length of segment : 363 time for calcul the mask position with numpy : 0.0006387233734130859 nb_pixel_total : 14883 time to create 1 rle with old method : 0.017294883728027344 length of segment : 182 time for calcul the mask position with numpy : 0.001058816909790039 nb_pixel_total : 31270 time to create 1 rle with old method : 0.03664541244506836 length of segment : 191 time for calcul the mask position with numpy : 0.0019316673278808594 nb_pixel_total : 35466 time to create 1 rle with old method : 0.04258012771606445 length of segment : 222 time for calcul the mask position with numpy : 0.001336812973022461 nb_pixel_total : 23701 time to create 1 rle with old method : 0.056673288345336914 length of segment : 283 time for calcul the mask position with numpy : 0.0005419254302978516 nb_pixel_total : 10425 time to create 1 rle with old method : 0.012766361236572266 length of segment : 121 time for calcul the mask position with numpy : 0.0004551410675048828 nb_pixel_total : 11181 time to create 1 rle with old method : 0.01614212989807129 length of segment : 88 time for calcul the mask position with numpy : 0.0025599002838134766 nb_pixel_total : 52467 time to create 1 rle with old method : 0.06160140037536621 length of segment : 338 time for calcul the mask position with numpy : 0.0006778240203857422 nb_pixel_total : 6932 time to create 1 rle with old method : 0.008816719055175781 length of segment : 119 time for calcul the mask position with numpy : 0.0006148815155029297 nb_pixel_total : 10594 time to create 1 rle with old method : 0.012442588806152344 length of segment : 180 time for calcul the mask position with numpy : 0.0031614303588867188 nb_pixel_total : 113528 time to create 1 rle with old method : 0.1481931209564209 length of segment : 698 time for calcul the mask position with numpy : 0.002866029739379883 nb_pixel_total : 61369 time to create 1 rle with old method : 0.0755620002746582 length of segment : 369 time for calcul the mask position with numpy : 0.0005931854248046875 nb_pixel_total : 14746 time to create 1 rle with old method : 0.016462087631225586 length of segment : 186 time for calcul the mask position with numpy : 0.0006625652313232422 nb_pixel_total : 11847 time to create 1 rle with old method : 0.013288497924804688 length of segment : 205 time for calcul the mask position with numpy : 0.0014908313751220703 nb_pixel_total : 28350 time to create 1 rle with old method : 0.048282623291015625 length of segment : 222 time for calcul the mask position with numpy : 0.0042896270751953125 nb_pixel_total : 108493 time to create 1 rle with old method : 0.14131927490234375 length of segment : 443 time for calcul the mask position with numpy : 0.0006005764007568359 nb_pixel_total : 15814 time to create 1 rle with old method : 0.018311262130737305 length of segment : 109 time for calcul the mask position with numpy : 0.0006117820739746094 nb_pixel_total : 17824 time to create 1 rle with old method : 0.02095627784729004 length of segment : 125 time for calcul the mask position with numpy : 0.0007908344268798828 nb_pixel_total : 19618 time to create 1 rle with old method : 0.022422313690185547 length of segment : 155 time for calcul the mask position with numpy : 0.0009849071502685547 nb_pixel_total : 26677 time to create 1 rle with old method : 0.031220674514770508 length of segment : 169 time for calcul the mask position with numpy : 0.0009372234344482422 nb_pixel_total : 22691 time to create 1 rle with old method : 0.025899171829223633 length of segment : 154 time for calcul the mask position with numpy : 0.0007691383361816406 nb_pixel_total : 22458 time to create 1 rle with old method : 0.026239395141601562 length of segment : 167 time for calcul the mask position with numpy : 0.0005605220794677734 nb_pixel_total : 13848 time to create 1 rle with old method : 0.016902446746826172 length of segment : 144 time for calcul the mask position with numpy : 0.0031585693359375 nb_pixel_total : 90401 time to create 1 rle with old method : 0.10709428787231445 length of segment : 308 time for calcul the mask position with numpy : 0.0004749298095703125 nb_pixel_total : 9568 time to create 1 rle with old method : 0.01140594482421875 length of segment : 111 time for calcul the mask position with numpy : 0.0006344318389892578 nb_pixel_total : 21786 time to create 1 rle with old method : 0.025279760360717773 length of segment : 219 time for calcul the mask position with numpy : 0.010448455810546875 nb_pixel_total : 223195 time to create 1 rle with new method : 0.01663517951965332 length of segment : 1156 time for calcul the mask position with numpy : 0.003034830093383789 nb_pixel_total : 74208 time to create 1 rle with old method : 0.0862267017364502 length of segment : 438 time for calcul the mask position with numpy : 0.00034737586975097656 nb_pixel_total : 13906 time to create 1 rle with old method : 0.016509056091308594 length of segment : 126 time for calcul the mask position with numpy : 0.0017807483673095703 nb_pixel_total : 38172 time to create 1 rle with old method : 0.04475712776184082 length of segment : 209 time for calcul the mask position with numpy : 0.0005056858062744141 nb_pixel_total : 11837 time to create 1 rle with old method : 0.013582468032836914 length of segment : 165 time for calcul the mask position with numpy : 0.00046133995056152344 nb_pixel_total : 20280 time to create 1 rle with old method : 0.023639917373657227 length of segment : 149 time for calcul the mask position with numpy : 0.0005245208740234375 nb_pixel_total : 14661 time to create 1 rle with old method : 0.016631603240966797 length of segment : 148 time for calcul the mask position with numpy : 0.0015358924865722656 nb_pixel_total : 49438 time to create 1 rle with old method : 0.05544424057006836 length of segment : 292 time for calcul the mask position with numpy : 0.00031280517578125 nb_pixel_total : 5330 time to create 1 rle with old method : 0.00611424446105957 length of segment : 104 time for calcul the mask position with numpy : 0.0016002655029296875 nb_pixel_total : 44296 time to create 1 rle with old method : 0.050826311111450195 length of segment : 379 time for calcul the mask position with numpy : 0.0012049674987792969 nb_pixel_total : 42621 time to create 1 rle with old method : 0.04849982261657715 length of segment : 347 time for calcul the mask position with numpy : 0.0007836818695068359 nb_pixel_total : 14739 time to create 1 rle with old method : 0.017158985137939453 length of segment : 161 time for calcul the mask position with numpy : 0.0007374286651611328 nb_pixel_total : 24932 time to create 1 rle with old method : 0.028858661651611328 length of segment : 258 time for calcul the mask position with numpy : 0.0005960464477539062 nb_pixel_total : 18688 time to create 1 rle with old method : 0.022766590118408203 length of segment : 152 time spent for convertir_results : 12.351437091827393 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 123 chid ids of type : 3594 Number RLEs to save : 35490 save missing photos in datou_result : time spend for datou_step_exec : 63.366116762161255 time spend to save output : 2.1134204864501953 total time spend for step 1 : 65.47953724861145 step2:crop_condition Wed May 28 16:51:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 123 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 92 About to insert : list_path_to_insert length 92 new photo from crops ! About to upload 92 photos upload in portfolio : 3736932 init cache_photo without model_param we have 92 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748443930_2135421 we have uploaded 92 photos in the portfolio 3736932 time of upload the photos Elapsed time : 25.875903367996216 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3736932 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748443959_2135421 we have uploaded 9 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.735273599624634 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 3736932 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748443969_2135421 we have uploaded 20 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.129932403564453 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748443977_2135421 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7460916042327881 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748443979_2135421 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6787528991699219 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1361519496, 1361518461, 1361518435, 1361518341, 1361518263, 1361517980, 1361517906, 1361517877, 1361517825] Looping around the photos to save general results len do output : 123 /1361530748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361530988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361519496', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518461', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518435', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518341', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518263', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517980', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517906', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517877', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517825', None, None, None, None, None, '2977366') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 378 time used for this insertion : 0.031000375747680664 save_final save missing photos in datou_result : time spend for datou_step_exec : 81.68573355674744 time spend to save output : 0.03613638877868652 total time spend for step 2 : 81.72186994552612 step3:rle_unique_nms_with_priority Wed May 28 16:52:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 123 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 15 nb_hashtags : 3 time to prepare the origin masks : 5.888030290603638 time for calcul the mask position with numpy : 0.42012524604797363 nb_pixel_total : 5209675 time to create 1 rle with new method : 0.7838146686553955 time for calcul the mask position with numpy : 0.025845050811767578 nb_pixel_total : 194971 time to create 1 rle with new method : 0.594989538192749 time for calcul the mask position with numpy : 0.03352713584899902 nb_pixel_total : 40274 time to create 1 rle with old method : 0.043904781341552734 time for calcul the mask position with numpy : 0.034409284591674805 nb_pixel_total : 168100 time to create 1 rle with new method : 0.5780966281890869 time for calcul the mask position with numpy : 0.03312492370605469 nb_pixel_total : 9174 time to create 1 rle with old method : 0.01013040542602539 time for calcul the mask position with numpy : 0.033797264099121094 nb_pixel_total : 70968 time to create 1 rle with old method : 0.07816696166992188 time for calcul the mask position with numpy : 0.03472137451171875 nb_pixel_total : 41794 time to create 1 rle with old method : 0.04605555534362793 time for calcul the mask position with numpy : 0.03403902053833008 nb_pixel_total : 12711 time to create 1 rle with old method : 0.01355433464050293 time for calcul the mask position with numpy : 0.0331883430480957 nb_pixel_total : 18418 time to create 1 rle with old method : 0.02034282684326172 time for calcul the mask position with numpy : 0.0695502758026123 nb_pixel_total : 1097983 time to create 1 rle with new method : 0.5096666812896729 time for calcul the mask position with numpy : 0.02941584587097168 nb_pixel_total : 52837 time to create 1 rle with old method : 0.055769920349121094 time for calcul the mask position with numpy : 0.027533531188964844 nb_pixel_total : 7622 time to create 1 rle with old method : 0.008054018020629883 time for calcul the mask position with numpy : 0.020303726196289062 nb_pixel_total : 36525 time to create 1 rle with old method : 0.03845977783203125 time for calcul the mask position with numpy : 0.019980907440185547 nb_pixel_total : 10032 time to create 1 rle with old method : 0.010354757308959961 time for calcul the mask position with numpy : 0.01966691017150879 nb_pixel_total : 35825 time to create 1 rle with old method : 0.0388185977935791 time for calcul the mask position with numpy : 0.019583463668823242 nb_pixel_total : 43331 time to create 1 rle with old method : 0.04600930213928223 create new chi : 3.8674042224884033 time to delete rle : 0.026371479034423828 batch 1 Loaded 31 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 11579 TO DO : save crop sub photo not yet done ! save time : 0.6810851097106934 nb_obj : 12 nb_hashtags : 3 time to prepare the origin masks : 4.4573915004730225 time for calcul the mask position with numpy : 0.4203455448150635 nb_pixel_total : 5559213 time to create 1 rle with new method : 0.7114048004150391 time for calcul the mask position with numpy : 0.022105932235717773 nb_pixel_total : 247248 time to create 1 rle with new method : 1.017378568649292 time for calcul the mask position with numpy : 0.03133392333984375 nb_pixel_total : 538656 time to create 1 rle with new method : 0.541743278503418 time for calcul the mask position with numpy : 0.02079486846923828 nb_pixel_total : 32357 time to create 1 rle with old method : 0.035718441009521484 time for calcul the mask position with numpy : 0.02035665512084961 nb_pixel_total : 25956 time to create 1 rle with old method : 0.02847743034362793 time for calcul the mask position with numpy : 0.02138662338256836 nb_pixel_total : 315742 time to create 1 rle with new method : 0.4253208637237549 time for calcul the mask position with numpy : 0.02121567726135254 nb_pixel_total : 17312 time to create 1 rle with old method : 0.018996715545654297 time for calcul the mask position with numpy : 0.0224764347076416 nb_pixel_total : 105080 time to create 1 rle with old method : 0.1150658130645752 time for calcul the mask position with numpy : 0.02138209342956543 nb_pixel_total : 13086 time to create 1 rle with old method : 0.014315366744995117 time for calcul the mask position with numpy : 0.020689725875854492 nb_pixel_total : 38826 time to create 1 rle with old method : 0.0435330867767334 time for calcul the mask position with numpy : 0.02086949348449707 nb_pixel_total : 42180 time to create 1 rle with old method : 0.04623913764953613 time for calcul the mask position with numpy : 0.03148841857910156 nb_pixel_total : 61226 time to create 1 rle with old method : 0.07579755783081055 time for calcul the mask position with numpy : 0.03418302536010742 nb_pixel_total : 53358 time to create 1 rle with old method : 0.06862235069274902 create new chi : 3.9582996368408203 time to delete rle : 0.0021791458129882812 batch 1 Loaded 25 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 11630 TO DO : save crop sub photo not yet done ! save time : 0.6962733268737793 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 5.27531886100769 time for calcul the mask position with numpy : 0.3977816104888916 nb_pixel_total : 6752603 time to create 1 rle with new method : 0.7701618671417236 time for calcul the mask position with numpy : 0.023082733154296875 nb_pixel_total : 24256 time to create 1 rle with old method : 0.027553558349609375 time for calcul the mask position with numpy : 0.02674579620361328 nb_pixel_total : 39124 time to create 1 rle with old method : 0.07119941711425781 time for calcul the mask position with numpy : 0.026837825775146484 nb_pixel_total : 22485 time to create 1 rle with old method : 0.025456905364990234 time for calcul the mask position with numpy : 0.022218704223632812 nb_pixel_total : 8821 time to create 1 rle with old method : 0.01011514663696289 time for calcul the mask position with numpy : 0.02312946319580078 nb_pixel_total : 25105 time to create 1 rle with old method : 0.0286102294921875 time for calcul the mask position with numpy : 0.02527475357055664 nb_pixel_total : 12593 time to create 1 rle with old method : 0.014407873153686523 time for calcul the mask position with numpy : 0.021622657775878906 nb_pixel_total : 14218 time to create 1 rle with old method : 0.01608729362487793 time for calcul the mask position with numpy : 0.021920442581176758 nb_pixel_total : 4023 time to create 1 rle with old method : 0.00490260124206543 time for calcul the mask position with numpy : 0.02146768569946289 nb_pixel_total : 31574 time to create 1 rle with old method : 0.03577828407287598 time for calcul the mask position with numpy : 0.021167993545532227 nb_pixel_total : 22011 time to create 1 rle with old method : 0.0249636173248291 time for calcul the mask position with numpy : 0.022757291793823242 nb_pixel_total : 43355 time to create 1 rle with old method : 0.049185991287231445 time for calcul the mask position with numpy : 0.02432703971862793 nb_pixel_total : 12131 time to create 1 rle with old method : 0.01374197006225586 time for calcul the mask position with numpy : 0.0233767032623291 nb_pixel_total : 13983 time to create 1 rle with old method : 0.015702009201049805 time for calcul the mask position with numpy : 0.020844221115112305 nb_pixel_total : 23958 time to create 1 rle with old method : 0.026627779006958008 create new chi : 1.8999841213226318 time to delete rle : 0.0011467933654785156 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 7670 TO DO : save crop sub photo not yet done ! save time : 0.48907470703125 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 4.138596773147583 time for calcul the mask position with numpy : 0.3504800796508789 nb_pixel_total : 6348754 time to create 1 rle with new method : 1.8871009349822998 time for calcul the mask position with numpy : 0.04081988334655762 nb_pixel_total : 33196 time to create 1 rle with old method : 0.03689455986022949 time for calcul the mask position with numpy : 0.020934104919433594 nb_pixel_total : 14117 time to create 1 rle with old method : 0.015619277954101562 time for calcul the mask position with numpy : 0.020118236541748047 nb_pixel_total : 22015 time to create 1 rle with old method : 0.023984432220458984 time for calcul the mask position with numpy : 0.020296096801757812 nb_pixel_total : 46407 time to create 1 rle with old method : 0.052397966384887695 time for calcul the mask position with numpy : 0.022765636444091797 nb_pixel_total : 180592 time to create 1 rle with new method : 0.5995757579803467 time for calcul the mask position with numpy : 0.022022247314453125 nb_pixel_total : 10648 time to create 1 rle with old method : 0.01155543327331543 time for calcul the mask position with numpy : 0.02149510383605957 nb_pixel_total : 20079 time to create 1 rle with old method : 0.021782398223876953 time for calcul the mask position with numpy : 0.021572589874267578 nb_pixel_total : 20657 time to create 1 rle with old method : 0.023250102996826172 time for calcul the mask position with numpy : 0.02205944061279297 nb_pixel_total : 56408 time to create 1 rle with old method : 0.06581568717956543 time for calcul the mask position with numpy : 0.023278236389160156 nb_pixel_total : 238513 time to create 1 rle with new method : 0.6078617572784424 time for calcul the mask position with numpy : 0.02212977409362793 nb_pixel_total : 58854 time to create 1 rle with old method : 0.06230044364929199 create new chi : 4.100813150405884 time to delete rle : 0.0012171268463134766 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++Number RLEs to save : 8920 TO DO : save crop sub photo not yet done ! save time : 0.5489802360534668 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 4.13985276222229 time for calcul the mask position with numpy : 0.5274143218994141 nb_pixel_total : 5266648 time to create 1 rle with new method : 0.6462500095367432 time for calcul the mask position with numpy : 0.022231101989746094 nb_pixel_total : 98455 time to create 1 rle with old method : 0.10850167274475098 time for calcul the mask position with numpy : 0.021275997161865234 nb_pixel_total : 95681 time to create 1 rle with old method : 0.10258936882019043 time for calcul the mask position with numpy : 0.021599769592285156 nb_pixel_total : 258225 time to create 1 rle with new method : 0.6455686092376709 time for calcul the mask position with numpy : 0.021277189254760742 nb_pixel_total : 40211 time to create 1 rle with old method : 0.044202566146850586 time for calcul the mask position with numpy : 0.02101421356201172 nb_pixel_total : 17484 time to create 1 rle with old method : 0.018784761428833008 time for calcul the mask position with numpy : 0.019822120666503906 nb_pixel_total : 18786 time to create 1 rle with old method : 0.020106077194213867 time for calcul the mask position with numpy : 0.020616531372070312 nb_pixel_total : 114801 time to create 1 rle with old method : 0.12314772605895996 time for calcul the mask position with numpy : 0.020616531372070312 nb_pixel_total : 102720 time to create 1 rle with old method : 0.11196732521057129 time for calcul the mask position with numpy : 0.02094411849975586 nb_pixel_total : 15727 time to create 1 rle with old method : 0.017102956771850586 time for calcul the mask position with numpy : 0.024654626846313477 nb_pixel_total : 677470 time to create 1 rle with new method : 0.664135217666626 time for calcul the mask position with numpy : 0.023067235946655273 nb_pixel_total : 344032 time to create 1 rle with new method : 0.8379762172698975 create new chi : 4.212256908416748 time to delete rle : 0.002785921096801758 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++Number RLEs to save : 12380 TO DO : save crop sub photo not yet done ! save time : 0.7331986427307129 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 6.096716642379761 time for calcul the mask position with numpy : 0.5313994884490967 nb_pixel_total : 6635869 time to create 1 rle with new method : 0.6041741371154785 time for calcul the mask position with numpy : 0.034043073654174805 nb_pixel_total : 22182 time to create 1 rle with old method : 0.023964643478393555 time for calcul the mask position with numpy : 0.03278183937072754 nb_pixel_total : 11612 time to create 1 rle with old method : 0.012418031692504883 time for calcul the mask position with numpy : 0.03659510612487793 nb_pixel_total : 17564 time to create 1 rle with old method : 0.028434276580810547 time for calcul the mask position with numpy : 0.03891587257385254 nb_pixel_total : 13686 time to create 1 rle with old method : 0.015355825424194336 time for calcul the mask position with numpy : 0.035794973373413086 nb_pixel_total : 16186 time to create 1 rle with old method : 0.01800251007080078 time for calcul the mask position with numpy : 0.03495359420776367 nb_pixel_total : 11967 time to create 1 rle with old method : 0.01300358772277832 time for calcul the mask position with numpy : 0.03341269493103027 nb_pixel_total : 18119 time to create 1 rle with old method : 0.02092599868774414 time for calcul the mask position with numpy : 0.034577369689941406 nb_pixel_total : 25801 time to create 1 rle with old method : 0.02860260009765625 time for calcul the mask position with numpy : 0.03338479995727539 nb_pixel_total : 13693 time to create 1 rle with old method : 0.014991283416748047 time for calcul the mask position with numpy : 0.034132957458496094 nb_pixel_total : 66336 time to create 1 rle with old method : 0.07326126098632812 time for calcul the mask position with numpy : 0.03193950653076172 nb_pixel_total : 45616 time to create 1 rle with old method : 0.04945492744445801 time for calcul the mask position with numpy : 0.029529094696044922 nb_pixel_total : 45517 time to create 1 rle with old method : 0.04974102973937988 time for calcul the mask position with numpy : 0.029349327087402344 nb_pixel_total : 33669 time to create 1 rle with old method : 0.037552595138549805 time for calcul the mask position with numpy : 0.021216869354248047 nb_pixel_total : 9456 time to create 1 rle with old method : 0.010273933410644531 time for calcul the mask position with numpy : 0.02285909652709961 nb_pixel_total : 18774 time to create 1 rle with old method : 0.020496129989624023 time for calcul the mask position with numpy : 0.021576881408691406 nb_pixel_total : 15772 time to create 1 rle with old method : 0.0172274112701416 time for calcul the mask position with numpy : 0.020969152450561523 nb_pixel_total : 28421 time to create 1 rle with old method : 0.031174421310424805 create new chi : 2.166483163833618 time to delete rle : 0.0022144317626953125 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 9080 TO DO : save crop sub photo not yet done ! save time : 0.5745036602020264 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 6.09897255897522 time for calcul the mask position with numpy : 0.46889734268188477 nb_pixel_total : 6317532 time to create 1 rle with new method : 0.4755101203918457 time for calcul the mask position with numpy : 0.024553537368774414 nb_pixel_total : 11847 time to create 1 rle with old method : 0.013475656509399414 time for calcul the mask position with numpy : 0.020636796951293945 nb_pixel_total : 14746 time to create 1 rle with old method : 0.0162353515625 time for calcul the mask position with numpy : 0.025011539459228516 nb_pixel_total : 61369 time to create 1 rle with old method : 0.06818771362304688 time for calcul the mask position with numpy : 0.021924734115600586 nb_pixel_total : 113528 time to create 1 rle with old method : 0.1241300106048584 time for calcul the mask position with numpy : 0.02235245704650879 nb_pixel_total : 10594 time to create 1 rle with old method : 0.011568307876586914 time for calcul the mask position with numpy : 0.01972031593322754 nb_pixel_total : 6932 time to create 1 rle with old method : 0.007362842559814453 time for calcul the mask position with numpy : 0.0202176570892334 nb_pixel_total : 52467 time to create 1 rle with old method : 0.05750393867492676 time for calcul the mask position with numpy : 0.021496295928955078 nb_pixel_total : 11181 time to create 1 rle with old method : 0.012307167053222656 time for calcul the mask position with numpy : 0.020723581314086914 nb_pixel_total : 10425 time to create 1 rle with old method : 0.01137685775756836 time for calcul the mask position with numpy : 0.021059513092041016 nb_pixel_total : 23701 time to create 1 rle with old method : 0.026036500930786133 time for calcul the mask position with numpy : 0.020421266555786133 nb_pixel_total : 35466 time to create 1 rle with old method : 0.03867602348327637 time for calcul the mask position with numpy : 0.03407096862792969 nb_pixel_total : 31270 time to create 1 rle with old method : 0.03506183624267578 time for calcul the mask position with numpy : 0.03698301315307617 nb_pixel_total : 14883 time to create 1 rle with old method : 0.016536474227905273 time for calcul the mask position with numpy : 0.03697943687438965 nb_pixel_total : 43604 time to create 1 rle with old method : 0.049146413803100586 time for calcul the mask position with numpy : 0.03261208534240723 nb_pixel_total : 78268 time to create 1 rle with old method : 0.08714890480041504 time for calcul the mask position with numpy : 0.022029399871826172 nb_pixel_total : 13636 time to create 1 rle with old method : 0.015392541885375977 time for calcul the mask position with numpy : 0.022740602493286133 nb_pixel_total : 198791 time to create 1 rle with new method : 0.5687694549560547 create new chi : 2.5881075859069824 time to delete rle : 0.001828908920288086 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 11660 TO DO : save crop sub photo not yet done ! save time : 0.7143948078155518 nb_obj : 18 nb_hashtags : 4 time to prepare the origin masks : 7.254488706588745 time for calcul the mask position with numpy : 0.5907237529754639 nb_pixel_total : 6271114 time to create 1 rle with new method : 0.7776498794555664 time for calcul the mask position with numpy : 0.01981496810913086 nb_pixel_total : 20280 time to create 1 rle with old method : 0.021903038024902344 time for calcul the mask position with numpy : 0.020309925079345703 nb_pixel_total : 11837 time to create 1 rle with old method : 0.012901067733764648 time for calcul the mask position with numpy : 0.02194380760192871 nb_pixel_total : 38172 time to create 1 rle with old method : 0.04130411148071289 time for calcul the mask position with numpy : 0.020752668380737305 nb_pixel_total : 13906 time to create 1 rle with old method : 0.01506352424621582 time for calcul the mask position with numpy : 0.020389080047607422 nb_pixel_total : 74208 time to create 1 rle with old method : 0.08001112937927246 time for calcul the mask position with numpy : 0.020493507385253906 nb_pixel_total : 223195 time to create 1 rle with new method : 0.5700242519378662 time for calcul the mask position with numpy : 0.02046680450439453 nb_pixel_total : 21786 time to create 1 rle with old method : 0.02324843406677246 time for calcul the mask position with numpy : 0.020578622817993164 nb_pixel_total : 9568 time to create 1 rle with old method : 0.010651826858520508 time for calcul the mask position with numpy : 0.021475791931152344 nb_pixel_total : 90401 time to create 1 rle with old method : 0.09521913528442383 time for calcul the mask position with numpy : 0.020089149475097656 nb_pixel_total : 13848 time to create 1 rle with old method : 0.014963388442993164 time for calcul the mask position with numpy : 0.02037978172302246 nb_pixel_total : 22458 time to create 1 rle with old method : 0.025345802307128906 time for calcul the mask position with numpy : 0.020977258682250977 nb_pixel_total : 22691 time to create 1 rle with old method : 0.02533435821533203 time for calcul the mask position with numpy : 0.020968198776245117 nb_pixel_total : 26677 time to create 1 rle with old method : 0.02977132797241211 time for calcul the mask position with numpy : 0.020525217056274414 nb_pixel_total : 19618 time to create 1 rle with old method : 0.020997285842895508 time for calcul the mask position with numpy : 0.02006053924560547 nb_pixel_total : 17824 time to create 1 rle with old method : 0.019408702850341797 time for calcul the mask position with numpy : 0.021045207977294922 nb_pixel_total : 15814 time to create 1 rle with old method : 0.0169680118560791 time for calcul the mask position with numpy : 0.0216214656829834 nb_pixel_total : 108493 time to create 1 rle with old method : 0.11759233474731445 time for calcul the mask position with numpy : 0.021380186080932617 nb_pixel_total : 28350 time to create 1 rle with old method : 0.03021860122680664 create new chi : 2.9719741344451904 time to delete rle : 0.0017061233520507812 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 11298 TO DO : save crop sub photo not yet done ! save time : 0.6861798763275146 nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 3.217700242996216 time for calcul the mask position with numpy : 0.599287748336792 nb_pixel_total : 6835535 time to create 1 rle with new method : 0.7328767776489258 time for calcul the mask position with numpy : 0.033204078674316406 nb_pixel_total : 18688 time to create 1 rle with old method : 0.019551753997802734 time for calcul the mask position with numpy : 0.019886493682861328 nb_pixel_total : 24932 time to create 1 rle with old method : 0.02733755111694336 time for calcul the mask position with numpy : 0.021043062210083008 nb_pixel_total : 14739 time to create 1 rle with old method : 0.016123056411743164 time for calcul the mask position with numpy : 0.02073502540588379 nb_pixel_total : 42621 time to create 1 rle with old method : 0.045528411865234375 time for calcul the mask position with numpy : 0.021581411361694336 nb_pixel_total : 44296 time to create 1 rle with old method : 0.04793143272399902 time for calcul the mask position with numpy : 0.01997232437133789 nb_pixel_total : 5330 time to create 1 rle with old method : 0.005651712417602539 time for calcul the mask position with numpy : 0.01996445655822754 nb_pixel_total : 49438 time to create 1 rle with old method : 0.05427980422973633 time for calcul the mask position with numpy : 0.021355867385864258 nb_pixel_total : 14661 time to create 1 rle with old method : 0.01636958122253418 create new chi : 1.780022382736206 time to delete rle : 0.0009624958038330078 batch 1 Loaded 17 chid ids of type : 3594 +++++++++Number RLEs to save : 5842 TO DO : save crop sub photo not yet done ! save time : 0.37998342514038086 map_output_result : {1361519496: (0.0, 'Should be the crop_list due to order', 0), 1361518461: (0.0, 'Should be the crop_list due to order', 0), 1361518435: (0.0, 'Should be the crop_list due to order', 0), 1361518341: (0.0, 'Should be the crop_list due to order', 0), 1361518263: (0.0, 'Should be the crop_list due to order', 0), 1361517980: (0.0, 'Should be the crop_list due to order', 0), 1361517906: (0.0, 'Should be the crop_list due to order', 0), 1361517877: (0.0, 'Should be the crop_list due to order', 0), 1361517825: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1361519496, 1361518461, 1361518435, 1361518341, 1361518263, 1361517980, 1361517906, 1361517877, 1361517825] Looping around the photos to save general results len do output : 9 /1361519496.Didn't retrieve data . /1361518461.Didn't retrieve data . /1361518435.Didn't retrieve data . /1361518341.Didn't retrieve data . /1361518263.Didn't retrieve data . /1361517980.Didn't retrieve data . /1361517906.Didn't retrieve data . /1361517877.Didn't retrieve data . /1361517825.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361519496', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518461', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518435', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518341', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518263', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517980', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517906', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517877', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517825', None, None, None, None, None, '2977366') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.013651847839355469 save_final save missing photos in datou_result : time spend for datou_step_exec : 80.60949277877808 time spend to save output : 0.014049291610717773 total time spend for step 3 : 80.6235420703888 step4:ventilate_hashtags_in_portfolio Wed May 28 16:54:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23420184 get user id for portfolio 23420184 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23420184 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','background','pehd','papier','flou','autre','metal','carton','pet_fonce','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23420184 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','background','pehd','papier','flou','autre','metal','carton','pet_fonce','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23420184 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','background','pehd','papier','flou','autre','metal','carton','pet_fonce','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23420277,23420278,23420279,23420280,23420281,23420282,23420283,23420284,23420285,23420286,23420287?tags=mal_croppe,background,pehd,papier,flou,autre,metal,carton,pet_fonce,pet_clair,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1361519496, 1361518461, 1361518435, 1361518341, 1361518263, 1361517980, 1361517906, 1361517877, 1361517825] Looping around the photos to save general results len do output : 1 /23420184. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361519496', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518461', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518435', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518341', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518263', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517980', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517906', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517877', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517825', None, None, None, None, None, '2977366') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013622045516967773 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5893325805664062 time spend to save output : 0.013985872268676758 total time spend for step 4 : 1.603318452835083 step5:final Wed May 28 16:54:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1361519496: ('0.13010143389917703',), 1361518461: ('0.13010143389917703',), 1361518435: ('0.13010143389917703',), 1361518341: ('0.13010143389917703',), 1361518263: ('0.13010143389917703',), 1361517980: ('0.13010143389917703',), 1361517906: ('0.13010143389917703',), 1361517877: ('0.13010143389917703',), 1361517825: ('0.13010143389917703',)} new output for save of step final : {1361519496: ('0.13010143389917703',), 1361518461: ('0.13010143389917703',), 1361518435: ('0.13010143389917703',), 1361518341: ('0.13010143389917703',), 1361518263: ('0.13010143389917703',), 1361517980: ('0.13010143389917703',), 1361517906: ('0.13010143389917703',), 1361517877: ('0.13010143389917703',), 1361517825: ('0.13010143389917703',)} [1361519496, 1361518461, 1361518435, 1361518341, 1361518263, 1361517980, 1361517906, 1361517877, 1361517825] Looping around the photos to save general results len do output : 9 /1361519496.Didn't retrieve data . /1361518461.Didn't retrieve data . /1361518435.Didn't retrieve data . /1361518341.Didn't retrieve data . /1361518263.Didn't retrieve data . /1361517980.Didn't retrieve data . /1361517906.Didn't retrieve data . /1361517877.Didn't retrieve data . /1361517825.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361519496', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518461', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518435', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518341', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518263', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517980', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517906', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517877', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517825', None, None, None, None, None, '2977366') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.012140035629272461 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11008262634277344 time spend to save output : 0.012616395950317383 total time spend for step 5 : 0.12269902229309082 step6:blur_detection Wed May 28 16:54:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38.jpg resize: (2160, 3264) 1361519496 3.9000552216967743 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48.jpg resize: (2160, 3264) 1361518461 -0.5114446595881225 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2.jpg resize: (2160, 3264) 1361518435 -1.8055104666062867 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14.jpg resize: (2160, 3264) 1361518341 0.40268312836633235 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4.jpg resize: (2160, 3264) 1361518263 0.7515695569807768 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d.jpg resize: (2160, 3264) 1361517980 -1.747942541220889 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea.jpg resize: (2160, 3264) 1361517906 -2.7047898981697385 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa.jpg resize: (2160, 3264) 1361517877 -1.7529359314637951 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac.jpg resize: (2160, 3264) 1361517825 -0.9887583322571764 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851749_0.png resize: (169, 324) 1361530748 -0.2627268423052592 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851750_0.png resize: (241, 251) 1361530750 -1.5833316444294805 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851751_0.png resize: (89, 134) 1361530751 1.7149158629953527 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851752_0.png resize: (227, 296) 1361530753 -0.49718114475655856 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851753_0.png resize: (64, 155) 1361530756 -1.1040083475057458 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851754_0.png resize: (274, 360) 1361530757 -0.02765821568783646 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851755_0.png resize: (1051, 1327) 1361530758 1.348732245607558 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851756_0.png resize: (168, 156) 1361530759 -0.838355829400988 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851757_0.png resize: (141, 154) 1361530760 -1.9407776034035862 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851759_0.png resize: (416, 274) 1361530761 -1.1958205240290676 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851760_0.png resize: (111, 115) 1361530762 -0.4865086266800096 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851761_0.png resize: (362, 998) 1361530763 -1.0096453959493281 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851763_0.png resize: (665, 537) 1361530764 0.3513877336475962 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851764_0.png resize: (240, 367) 1361530765 -1.6328463394981416 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851765_0.png resize: (322, 293) 1361530767 -0.5992127269367318 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851766_0.png resize: (392, 157) 1361530768 -1.0837807616614925 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851768_0.png resize: (110, 173) 1361530769 -1.0360962580955133 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851770_0.png resize: (165, 149) 1361530770 -0.6334502931435947 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851773_0.png resize: (202, 263) 1361530771 -1.2130138624499869 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851774_0.png resize: (1387, 670) 1361530772 -0.2398777545953153 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851776_0.png resize: (218, 191) 1361530775 -1.9056048800195877 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851778_0.png resize: (145, 117) 1361530776 -1.6659093080871845 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851779_0.png resize: (284, 388) 1361530777 -1.1237336681503267 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851781_0.png resize: (249, 214) 1361530779 -1.5847867756430736 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851782_0.png resize: (201, 211) 1361530780 -0.07579973374412347 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851783_0.png resize: (167, 159) 1361530781 -1.665148462207117 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851784_0.png resize: (66, 258) 1361530784 -0.18428140728757103 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851785_0.png resize: (189, 187) 1361530785 0.20184160880003182 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851786_0.png resize: (89, 128) 1361530786 -0.32042402086792615 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851787_0.png resize: (203, 196) 1361530788 -1.9434940176463342 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851790_0.png resize: (379, 296) 1361530789 0.055305528718796106 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851792_0.png resize: (273, 320) 1361530790 -1.414471229465693 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851793_0.png resize: (139, 194) 1361530791 -0.5501963857239305 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851795_0.png resize: (137, 119) 1361530792 -0.36299817706956067 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851796_0.png resize: (584, 529) 1361530793 -0.5628719179223619 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851797_0.png resize: (182, 347) 1361530794 -0.06634207667330218 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851799_0.png resize: (159, 199) 1361530795 -1.1732362501729707 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851800_0.png resize: (202, 237) 1361530796 0.9203690513641156 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851801_0.png resize: (908, 476) 1361530797 -0.9746038989604233 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851802_0.png resize: (1342, 1120) 1361530798 0.16105865181627854 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851803_0.png resize: (239, 103) 1361530799 -1.50510084222562 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851805_0.png resize: (501, 388) 1361530800 -1.0235600779291922 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851807_0.png resize: (225, 173) 1361530801 -0.7963526655985128 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851808_0.png resize: (123, 588) 1361530802 -1.6566833075909968 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851809_0.png resize: (475, 767) 1361530803 0.16791895600184023 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851810_0.png resize: (294, 508) 1361530804 -1.126719320460646 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851811_0.png resize: (287, 620) 1361530805 -1.3446069956573634 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851812_0.png resize: (179, 246) 1361530806 -1.0718238403969047 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851813_0.png resize: (148, 148) 1361530807 0.4569362511369701 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851814_0.png resize: (179, 142) 1361530808 -0.4184595484703516 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851815_0.png resize: (132, 126) 1361530810 -1.4861295322425003 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851817_0.png resize: (257, 319) 1361530811 -0.34407756593066574 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851820_0.png resize: (158, 136) 1361530812 -0.36075402837560877 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851822_0.png resize: (245, 108) 1361530813 -0.9853263338008976 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851823_0.png resize: (141, 131) 1361530814 0.04810154676088594 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851824_0.png resize: (232, 126) 1361530815 -1.0969369899645445 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851826_0.png resize: (185, 179) 1361530816 -0.612900426096035 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851827_0.png resize: (73, 231) 1361530817 -0.3003451014675073 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851829_0.png resize: (514, 641) 1361530818 -2.5387849746795594 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851830_0.png resize: (149, 167) 1361530819 -1.653521958374514 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851832_0.png resize: (363, 199) 1361530820 -1.2242543712867162 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851833_0.png resize: (182, 137) 1361530821 -1.4706439246742937 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851834_0.png resize: (190, 227) 1361530822 -1.002712342374583 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851837_0.png resize: (121, 125) 1361530823 -1.2529464642717396 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851839_0.png resize: (338, 224) 1361530824 -0.7509180057939524 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851840_0.png resize: (111, 124) 1361530825 -2.161489028889586 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851841_0.png resize: (168, 83) 1361530826 -0.8353516844970711 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851842_0.png resize: (617, 275) 1361530827 -1.4610610258430778 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851843_0.png resize: (339, 413) 1361530828 -2.403301022084278 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851844_0.png resize: (184, 109) 1361530829 -0.85968318407619 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851845_0.png resize: (205, 104) 1361530830 -1.2827168729913125 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851846_0.png resize: (222, 259) 1361530831 -0.782277237656801 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851847_0.png resize: (409, 396) 1361530832 -0.9772240272932776 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851848_0.png resize: (109, 183) 1361530833 -0.8108367420912092 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851849_0.png resize: (124, 168) 1361530834 0.2296711833692921 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851850_0.png resize: (148, 227) 1361530835 -1.504390439699591 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851853_0.png resize: (151, 207) 1361530837 -0.6232369401125086 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851855_0.png resize: (300, 491) 1361530839 -0.2101248196537698 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851856_0.png resize: (110, 104) 1361530841 0.4755322137253461 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851857_0.png resize: (218, 130) 1361530843 -1.294545260383047 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851858_0.png resize: (672, 691) 1361530845 -2.427880058692344 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851859_0.png resize: (430, 234) 1361530847 -1.732029358113645 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851860_0.png resize: (126, 135) 1361530849 0.8512195725677487 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851861_0.png resize: (182, 292) 1361530851 -1.2691825839710331 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851862_0.png resize: (164, 85) 1361530853 1.5081630784745497 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851864_0.png resize: (148, 165) 1361530855 -0.3564820869595845 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851865_0.png resize: (292, 330) 1361530857 -0.20533725177218115 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851866_0.png resize: (104, 73) 1361530859 -1.3511803273506278 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851868_0.png resize: (253, 241) 1361530860 -1.1817576885162897 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851869_0.png resize: (153, 151) 1361530861 -0.1656161564093278 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851870_0.png resize: (258, 124) 1361530862 -0.4956611915062735 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851871_0.png resize: (150, 179) 1361530863 0.2709232543471295 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851772_0.png resize: (180, 311) 1361530883 -0.947924440872944 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851777_0.png resize: (103, 166) 1361530884 1.0662126498700604 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851788_0.png resize: (247, 272) 1361530886 -1.2203615772550636 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851789_0.png resize: (203, 148) 1361530888 -0.9637575215895462 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851791_0.png resize: (715, 586) 1361530890 -0.9035476640322158 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851806_0.png resize: (172, 179) 1361530892 -0.3269696150886137 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851821_0.png resize: (204, 207) 1361530894 -1.2850206178775008 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851835_0.png resize: (183, 321) 1361530895 -1.5095144373275433 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851851_0.png resize: (151, 259) 1361530897 -1.4641876993558787 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851758_0.png resize: (197, 290) 1361530951 -0.9626518481431743 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851767_0.png resize: (135, 406) 1361530953 -0.05927219763880202 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851769_0.png resize: (270, 564) 1361530955 -3.216608389429254 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851771_0.png resize: (408, 960) 1361530957 -0.05072549435735561 treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851775_0.png resize: (695, 635) 1361530959 -1.0168784605595615 treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851780_0.png resize: (158, 184) 1361530960 -3.782518376769492 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851794_0.png resize: (247, 99) 1361530961 -0.46078711748468154 treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851798_0.png resize: (250, 112) 1361530962 -0.02154618223893104 treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851804_0.png resize: (396, 316) 1361530963 -0.6819979790473742 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851816_0.png resize: (267, 157) 1361530964 -2.6281184712486008 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851818_0.png resize: (311, 214) 1361530965 -1.3034371782773373 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851819_0.png resize: (336, 305) 1361530966 -2.1669029838192198 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851825_0.png resize: (125, 136) 1361530967 -0.020989029943025622 treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851828_0.png resize: (165, 151) 1361530968 0.7033681143171725 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851831_0.png resize: (410, 282) 1361530969 -2.014557683965615 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851836_0.png resize: (195, 243) 1361530970 -1.9803164545766865 treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851838_0.png resize: (85, 150) 1361530971 -2.27114363058916 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851852_0.png resize: (150, 244) 1361530972 -1.091431101147475 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851863_0.png resize: (142, 174) 1361530973 -0.09651658393175866 treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851867_0.png resize: (379, 140) 1361530974 -0.6468358502896195 treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851854_0.png resize: (144, 133) 1361530981 -1.9376638822368146 treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851762_0.png resize: (234, 231) 1361530988 -1.463407372017973 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 132 time used for this insertion : 0.017583847045898438 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 132 time used for this insertion : 0.028586626052856445 save missing photos in datou_result : time spend for datou_step_exec : 33.91992807388306 time spend to save output : 0.05124163627624512 total time spend for step 6 : 33.9711697101593 step7:brightness Wed May 28 16:54:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38.jpg treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48.jpg treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2.jpg treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14.jpg treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4.jpg treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d.jpg treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea.jpg treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa.jpg treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac.jpg treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851749_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851750_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851751_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851752_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851753_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851754_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851755_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851756_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851757_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851759_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851760_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851761_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851763_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851764_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851765_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851766_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851768_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851770_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851773_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851774_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851776_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851778_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851779_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851781_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851782_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851783_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851784_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851785_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851786_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851787_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851790_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851792_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851793_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851795_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851796_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851797_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851799_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851800_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851801_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851802_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851803_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851805_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851807_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851808_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851809_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851810_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851811_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851812_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851813_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851814_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851815_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851817_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851820_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851822_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851823_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851824_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851826_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851827_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851829_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851830_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851832_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851833_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851834_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851837_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851839_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851840_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851841_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851842_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851843_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851844_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851845_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851846_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851847_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851848_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851849_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851850_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851853_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851855_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851856_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851857_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851858_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851859_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851860_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851861_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851862_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851864_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851865_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851866_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851868_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851869_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851870_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851871_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851772_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851777_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851788_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851789_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851791_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851806_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851821_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851835_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851851_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851758_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851767_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851769_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851771_0.png treat image : temp/1748443830_2135421_1361518461_b7c2b9d3e7f2ed94e654a26980f8ca48_rle_crop_3814851775_0.png treat image : temp/1748443830_2135421_1361518435_8087c66998c8565fbe81a2c8e398b4d2_rle_crop_3814851780_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851794_0.png treat image : temp/1748443830_2135421_1361518341_ebd2a1cc30f79d826d500b936da16c14_rle_crop_3814851798_0.png treat image : temp/1748443830_2135421_1361518263_a36b637e21c25b007b132e74cdab88e4_rle_crop_3814851804_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851816_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851818_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851819_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851825_0.png treat image : temp/1748443830_2135421_1361517980_f82471b27524921b1d1f57ea9ae42e7d_rle_crop_3814851828_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851831_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851836_0.png treat image : temp/1748443830_2135421_1361517906_5662e9b107cbd914c197b4abd2f0ebea_rle_crop_3814851838_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851852_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851863_0.png treat image : temp/1748443830_2135421_1361517825_571e60e78df31143d455641071ebafac_rle_crop_3814851867_0.png treat image : temp/1748443830_2135421_1361517877_010de0ad175af8b316dc30b16daadefa_rle_crop_3814851854_0.png treat image : temp/1748443830_2135421_1361519496_414da59d9d406d924996258c16d85b38_rle_crop_3814851762_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 132 time used for this insertion : 0.020420312881469727 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 132 time used for this insertion : 0.03734326362609863 save missing photos in datou_result : time spend for datou_step_exec : 8.176944971084595 time spend to save output : 0.06574773788452148 total time spend for step 7 : 8.242692708969116 step8:velours_tree Wed May 28 16:55:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.08133625984191895 time spend to save output : 7.700920104980469e-05 total time spend for step 8 : 0.08141326904296875 step9:send_mail_cod Wed May 28 16:55:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23420184_28-05-2025_16_55_04.pdf 23420277 imagette234202771748444104 23420278 imagette234202781748444104 23420279 change filename to text .imagette234202791748444104 23420280 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234202801748444104 23420281 imagette234202811748444105 23420282 change filename to text .imagette234202821748444105 23420283 imagette234202831748444105 23420284 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234202841748444105 23420285 imagette234202851748444106 23420286 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234202861748444106 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23420184 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23420277,23420278,23420279,23420280,23420281,23420282,23420283,23420284,23420285,23420286,23420287?tags=mal_croppe,background,pehd,papier,flou,autre,metal,carton,pet_fonce,pet_clair,environnement args[1361519496] : ((1361519496, 3.9000552216967743, 492688767), (1361519496, -0.03940429671675207, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361518461] : ((1361518461, -0.5114446595881225, 492688767), (1361518461, 0.3473436284510947, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361518435] : ((1361518435, -1.8055104666062867, 492688767), (1361518435, 0.0880142444264624, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361518341] : ((1361518341, 0.40268312836633235, 492688767), (1361518341, 0.44499635469621546, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361518263] : ((1361518263, 0.7515695569807768, 492688767), (1361518263, 0.09472560977163727, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361517980] : ((1361517980, -1.747942541220889, 492688767), (1361517980, 0.3766742419383597, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361517906] : ((1361517906, -2.7047898981697385, 492609224), (1361517906, 0.13983287776306616, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361517877] : ((1361517877, -1.7529359314637951, 492688767), (1361517877, 0.15165030975601607, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com args[1361517825] : ((1361517825, -0.9887583322571764, 492688767), (1361517825, 0.06301149142856308, 2107752395), '0.13010143389917703') We are sending mail with results at report@fotonower.com refus_total : 0.13010143389917703 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23420184 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23420184_28-05-2025_16_55_04.pdf results_Auto_P23420184_28-05-2025_16_55_04.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23420184_28-05-2025_16_55_04.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23420184','results_Auto_P23420184_28-05-2025_16_55_04.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23420184_28-05-2025_16_55_04.pdf','pdf','','0.51','0.13010143389917703') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23420184

https://www.fotonower.com/image?json=false&list_photos_id=1361519496
La photo est trop floue, merci de reprendre une photo.(avec le score = 3.9000552216967743)
https://www.fotonower.com/image?json=false&list_photos_id=1361518461
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361518435
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361518341
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361518263
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361517980
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361517906
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361517877
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361517825
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 13.01%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pehd: https://www.fotonower.com/view/23420279?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23420280?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23420282?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23420284?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23420286?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23420184_28-05-2025_16_55_04.pdf.

Lien vers velours :https://www.fotonower.com/velours/23420277,23420278,23420279,23420280,23420281,23420282,23420283,23420284,23420285,23420286,23420287?tags=mal_croppe,background,pehd,papier,flou,autre,metal,carton,pet_fonce,pet_clair,environnement.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 28 May 2025 14:55:10 GMT Content-Length: 0 Connection: close X-Message-Id: izMFiVRUTYiuG2cy7vLGGg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1361519496, 1361518461, 1361518435, 1361518341, 1361518263, 1361517980, 1361517906, 1361517877, 1361517825] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361519496', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518461', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518435', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518341', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518263', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517980', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517906', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517877', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517825', None, None, None, None, None, '2977366') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.014002084732055664 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.409694194793701 time spend to save output : 0.014255285263061523 total time spend for step 9 : 6.423949480056763 step10:split_time_score Wed May 28 16:55:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 9),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 28052025 23420184 Nombre de photos uploadées : 9 / 23040 (0%) 28052025 23420184 Nombre de photos taguées (types de déchets): 0 / 9 (0%) 28052025 23420184 Nombre de photos taguées (volume) : 0 / 9 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 4.5299530029296875e-06 ????????? elapsed_time : fill_and_build_computed_from_old_data 0.0005319118499755859 elapsed_time : insert_dashboard_record_day_entry 0.02664780616760254 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23412052 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23409912 order by id desc limit 1 Qualite : 0.09322079815722585 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23412055_28-05-2025_12_13_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23412055 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23412055 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23418628 order by id desc limit 1 Qualite : 0.0733784062670207 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23415307_28-05-2025_13_24_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23415307 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23415307 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23418653 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23418657 order by id desc limit 1 Qualite : 0.11586197538428952 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23418660_28-05-2025_15_58_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23418660 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23418660 AND mptpi.`type`=3594 To do Qualite : 0.13010143389917703 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23420184_28-05-2025_16_55_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23420184 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23420184 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'28052025': {'nb_upload': 9, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1361519496, 1361518461, 1361518435, 1361518341, 1361518263, 1361517980, 1361517906, 1361517877, 1361517825] Looping around the photos to save general results len do output : 1 /23420184Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361519496', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518461', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518435', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518341', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361518263', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517980', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517906', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517877', None, None, None, None, None, '2977366') ('3318', None, None, None, None, None, None, None, '2977366') ('3318', '23420184', '1361517825', None, None, None, None, None, '2977366') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.01367497444152832 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.949738025665283 time spend to save output : 0.013862133026123047 total time spend for step 10 : 4.963600158691406 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 9 set_done_treatment 126.06user 109.39system 4:49.30elapsed 81%CPU (0avgtext+0avgdata 4731564maxresident)k 505096inputs+67360outputs (38major+11315128minor)pagefaults 0swaps