python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 127184 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2577803'] with mtr_portfolio_ids : ['20462119'] and first list_photo_ids : [] new path : /proc/127184/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.641465663909912 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Feb 11 21:40:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2904 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-11 21:40:32.026105: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-11 21:40:32.051293: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-11 21:40:32.053305: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fc900000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-11 21:40:32.053340: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-11 21:40:32.057406: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-11 21:40:32.299332: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x37be0960 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-11 21:40:32.299388: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-11 21:40:32.300239: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-11 21:40:32.304267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 21:40:32.308095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 21:40:32.311128: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-11 21:40:32.312042: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-11 21:40:32.315033: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-11 21:40:32.316374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-11 21:40:32.321899: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 21:40:32.323211: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-11 21:40:32.323314: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 21:40:32.324003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-11 21:40:32.324023: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-11 21:40:32.324034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-11 21:40:32.324944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2452 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-11 21:40:32.642356: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-11 21:40:32.642443: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 21:40:32.642470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 21:40:32.642496: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-11 21:40:32.642520: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-11 21:40:32.642544: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-11 21:40:32.642568: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-11 21:40:32.642593: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 21:40:32.643946: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-11 21:40:32.644969: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-11 21:40:32.645002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 21:40:32.645017: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 21:40:32.645031: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-11 21:40:32.645045: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-11 21:40:32.645059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-11 21:40:32.645072: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-11 21:40:32.645086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 21:40:32.645886: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-11 21:40:32.645912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-11 21:40:32.645920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-11 21:40:32.645927: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-11 21:40:32.646773: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2452 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-11 21:40:42.280988: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 21:40:42.461959: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 21:40:43.898151: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:43.921229: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:43.928900: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:43.928972: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:43.983325: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:43.983369: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:44.026772: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:44.026815: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:44.079376: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:44.079421: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-11 21:40:44.082253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.40G (1499332608 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.082952: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.26G (1349399296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.401860: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-02-11 21:40:44.446362: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.448351: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.449631: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.450456: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.461312: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.462010: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.462743: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.463427: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.464122: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.465249: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.480283: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.480945: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.504963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.505034: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-11 21:40:44.505666: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.506258: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.513625: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.514246: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.522294: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.522913: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.539950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.540751: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.541535: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.542369: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.546773: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.547437: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.548083: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.549192: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.550437: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.560680: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.561565: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.570904: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.571560: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.572197: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.572816: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.573443: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-11 21:40:44.574059: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.90G (2036203520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 42 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 141.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 143.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 142.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 32 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 39 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 142.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 140.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 32 Detection mask done ! Trying to reset tf kernel 128099 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 912 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1558 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0018124580383300781 nb_pixel_total : 35506 time to create 1 rle with old method : 0.05474734306335449 length of segment : 219 time for calcul the mask position with numpy : 0.0011587142944335938 nb_pixel_total : 22129 time to create 1 rle with old method : 0.026947975158691406 length of segment : 234 time for calcul the mask position with numpy : 0.00046443939208984375 nb_pixel_total : 11737 time to create 1 rle with old method : 0.014961719512939453 length of segment : 122 time for calcul the mask position with numpy : 0.0013186931610107422 nb_pixel_total : 36070 time to create 1 rle with old method : 0.04634737968444824 length of segment : 213 time for calcul the mask position with numpy : 0.002887248992919922 nb_pixel_total : 79287 time to create 1 rle with old method : 0.09023404121398926 length of segment : 396 time for calcul the mask position with numpy : 0.0012736320495605469 nb_pixel_total : 33580 time to create 1 rle with old method : 0.04109835624694824 length of segment : 216 time for calcul the mask position with numpy : 0.013910770416259766 nb_pixel_total : 187402 time to create 1 rle with new method : 0.031022071838378906 length of segment : 533 time for calcul the mask position with numpy : 0.0008018016815185547 nb_pixel_total : 22601 time to create 1 rle with old method : 0.02657938003540039 length of segment : 182 time for calcul the mask position with numpy : 0.0008411407470703125 nb_pixel_total : 21415 time to create 1 rle with old method : 0.0269315242767334 length of segment : 190 time for calcul the mask position with numpy : 0.0008883476257324219 nb_pixel_total : 23818 time to create 1 rle with old method : 0.034804344177246094 length of segment : 261 time for calcul the mask position with numpy : 0.00044465065002441406 nb_pixel_total : 8584 time to create 1 rle with old method : 0.010113239288330078 length of segment : 146 time for calcul the mask position with numpy : 0.00041294097900390625 nb_pixel_total : 11008 time to create 1 rle with old method : 0.013307332992553711 length of segment : 107 time for calcul the mask position with numpy : 0.0024569034576416016 nb_pixel_total : 56514 time to create 1 rle with old method : 0.07900333404541016 length of segment : 335 time for calcul the mask position with numpy : 0.00044035911560058594 nb_pixel_total : 7002 time to create 1 rle with old method : 0.008476018905639648 length of segment : 78 time for calcul the mask position with numpy : 0.0007967948913574219 nb_pixel_total : 14572 time to create 1 rle with old method : 0.019080400466918945 length of segment : 157 time for calcul the mask position with numpy : 0.0022351741790771484 nb_pixel_total : 52983 time to create 1 rle with old method : 0.06198406219482422 length of segment : 335 time for calcul the mask position with numpy : 0.0010342597961425781 nb_pixel_total : 23496 time to create 1 rle with old method : 0.04580330848693848 length of segment : 264 time for calcul the mask position with numpy : 0.006524085998535156 nb_pixel_total : 167652 time to create 1 rle with new method : 0.024389028549194336 length of segment : 401 time for calcul the mask position with numpy : 0.0007236003875732422 nb_pixel_total : 13193 time to create 1 rle with old method : 0.01934194564819336 length of segment : 184 time for calcul the mask position with numpy : 0.0012705326080322266 nb_pixel_total : 26833 time to create 1 rle with old method : 0.03162956237792969 length of segment : 225 time for calcul the mask position with numpy : 0.0006496906280517578 nb_pixel_total : 13084 time to create 1 rle with old method : 0.01636338233947754 length of segment : 144 time for calcul the mask position with numpy : 0.0007288455963134766 nb_pixel_total : 25374 time to create 1 rle with old method : 0.030283451080322266 length of segment : 198 time for calcul the mask position with numpy : 0.0008347034454345703 nb_pixel_total : 13036 time to create 1 rle with old method : 0.014936208724975586 length of segment : 192 time for calcul the mask position with numpy : 0.007508516311645508 nb_pixel_total : 221815 time to create 1 rle with new method : 0.025753259658813477 length of segment : 847 time for calcul the mask position with numpy : 0.0038909912109375 nb_pixel_total : 87681 time to create 1 rle with old method : 0.11306095123291016 length of segment : 495 time for calcul the mask position with numpy : 0.0013172626495361328 nb_pixel_total : 25241 time to create 1 rle with old method : 0.032175540924072266 length of segment : 258 time for calcul the mask position with numpy : 0.0012001991271972656 nb_pixel_total : 38105 time to create 1 rle with old method : 0.042563676834106445 length of segment : 254 time for calcul the mask position with numpy : 0.0026857852935791016 nb_pixel_total : 127258 time to create 1 rle with old method : 0.14790916442871094 length of segment : 326 time for calcul the mask position with numpy : 0.0005602836608886719 nb_pixel_total : 13634 time to create 1 rle with old method : 0.025180816650390625 length of segment : 140 time for calcul the mask position with numpy : 0.004419565200805664 nb_pixel_total : 114298 time to create 1 rle with old method : 0.14128637313842773 length of segment : 527 time for calcul the mask position with numpy : 0.0011751651763916016 nb_pixel_total : 35338 time to create 1 rle with old method : 0.04062056541442871 length of segment : 371 time for calcul the mask position with numpy : 0.003630399703979492 nb_pixel_total : 112434 time to create 1 rle with old method : 0.12356162071228027 length of segment : 522 time for calcul the mask position with numpy : 0.0003685951232910156 nb_pixel_total : 11509 time to create 1 rle with old method : 0.012827634811401367 length of segment : 164 time for calcul the mask position with numpy : 0.0013003349304199219 nb_pixel_total : 48638 time to create 1 rle with old method : 0.05436372756958008 length of segment : 222 time for calcul the mask position with numpy : 0.0016531944274902344 nb_pixel_total : 46443 time to create 1 rle with old method : 0.05857253074645996 length of segment : 440 time for calcul the mask position with numpy : 0.0021708011627197266 nb_pixel_total : 36992 time to create 1 rle with old method : 0.04164433479309082 length of segment : 476 time for calcul the mask position with numpy : 0.002226591110229492 nb_pixel_total : 120674 time to create 1 rle with old method : 0.13270831108093262 length of segment : 302 time for calcul the mask position with numpy : 0.0006871223449707031 nb_pixel_total : 13636 time to create 1 rle with old method : 0.01517176628112793 length of segment : 140 time for calcul the mask position with numpy : 0.0017135143280029297 nb_pixel_total : 42210 time to create 1 rle with old method : 0.047273874282836914 length of segment : 324 time for calcul the mask position with numpy : 0.0010199546813964844 nb_pixel_total : 12681 time to create 1 rle with old method : 0.014347553253173828 length of segment : 189 time for calcul the mask position with numpy : 0.002404451370239258 nb_pixel_total : 37568 time to create 1 rle with old method : 0.04771852493286133 length of segment : 259 time for calcul the mask position with numpy : 0.008488655090332031 nb_pixel_total : 213252 time to create 1 rle with new method : 0.013953208923339844 length of segment : 843 time for calcul the mask position with numpy : 0.0029554367065429688 nb_pixel_total : 57650 time to create 1 rle with old method : 0.06317019462585449 length of segment : 301 time for calcul the mask position with numpy : 0.004590749740600586 nb_pixel_total : 186331 time to create 1 rle with new method : 0.012584447860717773 length of segment : 611 time for calcul the mask position with numpy : 0.0016062259674072266 nb_pixel_total : 27116 time to create 1 rle with old method : 0.03418779373168945 length of segment : 412 time for calcul the mask position with numpy : 0.003450155258178711 nb_pixel_total : 71501 time to create 1 rle with old method : 0.09514307975769043 length of segment : 489 time for calcul the mask position with numpy : 0.0019469261169433594 nb_pixel_total : 51322 time to create 1 rle with old method : 0.07401227951049805 length of segment : 448 time for calcul the mask position with numpy : 0.0027923583984375 nb_pixel_total : 75866 time to create 1 rle with old method : 0.08201360702514648 length of segment : 484 time for calcul the mask position with numpy : 0.0004017353057861328 nb_pixel_total : 10471 time to create 1 rle with old method : 0.011625289916992188 length of segment : 152 time for calcul the mask position with numpy : 0.0004055500030517578 nb_pixel_total : 12991 time to create 1 rle with old method : 0.015209436416625977 length of segment : 143 time for calcul the mask position with numpy : 0.008987903594970703 nb_pixel_total : 298525 time to create 1 rle with new method : 0.01896500587463379 length of segment : 596 time for calcul the mask position with numpy : 0.004932403564453125 nb_pixel_total : 151576 time to create 1 rle with new method : 0.015051841735839844 length of segment : 935 time for calcul the mask position with numpy : 0.0012984275817871094 nb_pixel_total : 33523 time to create 1 rle with old method : 0.03599858283996582 length of segment : 330 time for calcul the mask position with numpy : 0.0008616447448730469 nb_pixel_total : 28808 time to create 1 rle with old method : 0.03336381912231445 length of segment : 173 time for calcul the mask position with numpy : 0.0005714893341064453 nb_pixel_total : 21676 time to create 1 rle with old method : 0.023988008499145508 length of segment : 201 time for calcul the mask position with numpy : 0.001140594482421875 nb_pixel_total : 47616 time to create 1 rle with old method : 0.05362057685852051 length of segment : 367 time for calcul the mask position with numpy : 0.0030074119567871094 nb_pixel_total : 77669 time to create 1 rle with old method : 0.09170031547546387 length of segment : 396 time for calcul the mask position with numpy : 0.0012180805206298828 nb_pixel_total : 19614 time to create 1 rle with old method : 0.02270960807800293 length of segment : 163 time for calcul the mask position with numpy : 0.0024755001068115234 nb_pixel_total : 75210 time to create 1 rle with old method : 0.08991193771362305 length of segment : 347 time for calcul the mask position with numpy : 0.002675294876098633 nb_pixel_total : 70131 time to create 1 rle with old method : 0.07775163650512695 length of segment : 362 time for calcul the mask position with numpy : 0.002366781234741211 nb_pixel_total : 65147 time to create 1 rle with old method : 0.07125735282897949 length of segment : 365 time for calcul the mask position with numpy : 0.0008187294006347656 nb_pixel_total : 13822 time to create 1 rle with old method : 0.01580190658569336 length of segment : 152 time for calcul the mask position with numpy : 0.0018734931945800781 nb_pixel_total : 45237 time to create 1 rle with old method : 0.04947686195373535 length of segment : 373 time for calcul the mask position with numpy : 0.0005919933319091797 nb_pixel_total : 11010 time to create 1 rle with old method : 0.012453556060791016 length of segment : 102 time for calcul the mask position with numpy : 0.0005257129669189453 nb_pixel_total : 10310 time to create 1 rle with old method : 0.011935949325561523 length of segment : 109 time for calcul the mask position with numpy : 0.001049041748046875 nb_pixel_total : 12323 time to create 1 rle with old method : 0.022667884826660156 length of segment : 194 time for calcul the mask position with numpy : 0.0013775825500488281 nb_pixel_total : 35131 time to create 1 rle with old method : 0.0383303165435791 length of segment : 208 time for calcul the mask position with numpy : 0.0030024051666259766 nb_pixel_total : 79797 time to create 1 rle with old method : 0.09190487861633301 length of segment : 325 time for calcul the mask position with numpy : 0.0005147457122802734 nb_pixel_total : 5094 time to create 1 rle with old method : 0.006279706954956055 length of segment : 77 time for calcul the mask position with numpy : 0.0018002986907958984 nb_pixel_total : 36647 time to create 1 rle with old method : 0.04059648513793945 length of segment : 209 time for calcul the mask position with numpy : 0.0030035972595214844 nb_pixel_total : 47820 time to create 1 rle with old method : 0.058055877685546875 length of segment : 354 time for calcul the mask position with numpy : 0.002305269241333008 nb_pixel_total : 49160 time to create 1 rle with old method : 0.05417132377624512 length of segment : 239 time for calcul the mask position with numpy : 0.001993417739868164 nb_pixel_total : 37493 time to create 1 rle with old method : 0.04451417922973633 length of segment : 307 time for calcul the mask position with numpy : 0.0024979114532470703 nb_pixel_total : 44102 time to create 1 rle with old method : 0.051483154296875 length of segment : 343 time for calcul the mask position with numpy : 0.0016021728515625 nb_pixel_total : 22440 time to create 1 rle with old method : 0.029163122177124023 length of segment : 370 time for calcul the mask position with numpy : 0.0006425380706787109 nb_pixel_total : 24033 time to create 1 rle with old method : 0.02838420867919922 length of segment : 178 time for calcul the mask position with numpy : 0.0034859180450439453 nb_pixel_total : 49155 time to create 1 rle with old method : 0.05686306953430176 length of segment : 426 time for calcul the mask position with numpy : 0.0012927055358886719 nb_pixel_total : 33275 time to create 1 rle with old method : 0.03824663162231445 length of segment : 344 time for calcul the mask position with numpy : 0.0016131401062011719 nb_pixel_total : 38203 time to create 1 rle with old method : 0.05220389366149902 length of segment : 260 time for calcul the mask position with numpy : 0.0008683204650878906 nb_pixel_total : 24152 time to create 1 rle with old method : 0.029216289520263672 length of segment : 202 time for calcul the mask position with numpy : 0.0003504753112792969 nb_pixel_total : 5815 time to create 1 rle with old method : 0.006865024566650391 length of segment : 67 time for calcul the mask position with numpy : 0.0007796287536621094 nb_pixel_total : 21854 time to create 1 rle with old method : 0.024967432022094727 length of segment : 167 time for calcul the mask position with numpy : 0.0010423660278320312 nb_pixel_total : 25416 time to create 1 rle with old method : 0.029963254928588867 length of segment : 157 time for calcul the mask position with numpy : 0.0022971630096435547 nb_pixel_total : 72045 time to create 1 rle with old method : 0.07930827140808105 length of segment : 484 time for calcul the mask position with numpy : 0.0014834403991699219 nb_pixel_total : 27197 time to create 1 rle with old method : 0.03211522102355957 length of segment : 249 time for calcul the mask position with numpy : 0.0008716583251953125 nb_pixel_total : 20334 time to create 1 rle with old method : 0.023905277252197266 length of segment : 104 time for calcul the mask position with numpy : 0.0014128684997558594 nb_pixel_total : 35760 time to create 1 rle with old method : 0.04027533531188965 length of segment : 265 time for calcul the mask position with numpy : 0.0009427070617675781 nb_pixel_total : 21765 time to create 1 rle with old method : 0.02415180206298828 length of segment : 131 time for calcul the mask position with numpy : 0.001132965087890625 nb_pixel_total : 24656 time to create 1 rle with old method : 0.028962135314941406 length of segment : 230 time for calcul the mask position with numpy : 0.0027747154235839844 nb_pixel_total : 32366 time to create 1 rle with old method : 0.03702378273010254 length of segment : 207 time for calcul the mask position with numpy : 0.0017859935760498047 nb_pixel_total : 43615 time to create 1 rle with old method : 0.047350406646728516 length of segment : 309 time for calcul the mask position with numpy : 0.0007960796356201172 nb_pixel_total : 13785 time to create 1 rle with old method : 0.016356945037841797 length of segment : 176 time for calcul the mask position with numpy : 0.0007708072662353516 nb_pixel_total : 16936 time to create 1 rle with old method : 0.023779630661010742 length of segment : 120 time for calcul the mask position with numpy : 0.002265453338623047 nb_pixel_total : 46168 time to create 1 rle with old method : 0.061276912689208984 length of segment : 256 time for calcul the mask position with numpy : 0.0008645057678222656 nb_pixel_total : 17072 time to create 1 rle with old method : 0.01954960823059082 length of segment : 133 time for calcul the mask position with numpy : 0.002360820770263672 nb_pixel_total : 37661 time to create 1 rle with old method : 0.04179048538208008 length of segment : 335 time for calcul the mask position with numpy : 0.003037691116333008 nb_pixel_total : 75434 time to create 1 rle with old method : 0.08490920066833496 length of segment : 352 time for calcul the mask position with numpy : 0.0012538433074951172 nb_pixel_total : 18493 time to create 1 rle with old method : 0.02296757698059082 length of segment : 196 time for calcul the mask position with numpy : 0.0022919178009033203 nb_pixel_total : 57622 time to create 1 rle with old method : 0.07394266128540039 length of segment : 455 time for calcul the mask position with numpy : 0.0004203319549560547 nb_pixel_total : 7577 time to create 1 rle with old method : 0.00912332534790039 length of segment : 121 time for calcul the mask position with numpy : 0.0012884140014648438 nb_pixel_total : 32504 time to create 1 rle with old method : 0.03823089599609375 length of segment : 240 time for calcul the mask position with numpy : 0.002070903778076172 nb_pixel_total : 33901 time to create 1 rle with old method : 0.05950736999511719 length of segment : 256 time for calcul the mask position with numpy : 0.001287221908569336 nb_pixel_total : 16056 time to create 1 rle with old method : 0.02940201759338379 length of segment : 285 time for calcul the mask position with numpy : 0.0015802383422851562 nb_pixel_total : 32680 time to create 1 rle with old method : 0.03680300712585449 length of segment : 273 time for calcul the mask position with numpy : 0.0019466876983642578 nb_pixel_total : 51462 time to create 1 rle with old method : 0.058191776275634766 length of segment : 296 time for calcul the mask position with numpy : 0.001384735107421875 nb_pixel_total : 28514 time to create 1 rle with old method : 0.045697689056396484 length of segment : 178 time for calcul the mask position with numpy : 0.001725912094116211 nb_pixel_total : 39141 time to create 1 rle with old method : 0.04931330680847168 length of segment : 286 time for calcul the mask position with numpy : 0.001728057861328125 nb_pixel_total : 34311 time to create 1 rle with old method : 0.04001045227050781 length of segment : 263 time for calcul the mask position with numpy : 0.001306295394897461 nb_pixel_total : 21131 time to create 1 rle with old method : 0.02387237548828125 length of segment : 220 time for calcul the mask position with numpy : 0.001477956771850586 nb_pixel_total : 27699 time to create 1 rle with old method : 0.037308692932128906 length of segment : 193 time for calcul the mask position with numpy : 0.0006394386291503906 nb_pixel_total : 10520 time to create 1 rle with old method : 0.013416051864624023 length of segment : 146 time for calcul the mask position with numpy : 0.0023572444915771484 nb_pixel_total : 37929 time to create 1 rle with old method : 0.045766353607177734 length of segment : 214 time for calcul the mask position with numpy : 0.0015671253204345703 nb_pixel_total : 27427 time to create 1 rle with old method : 0.03218340873718262 length of segment : 297 time for calcul the mask position with numpy : 0.00025153160095214844 nb_pixel_total : 5684 time to create 1 rle with old method : 0.006664276123046875 length of segment : 100 time for calcul the mask position with numpy : 0.005148649215698242 nb_pixel_total : 96023 time to create 1 rle with old method : 0.11335563659667969 length of segment : 608 time for calcul the mask position with numpy : 0.0008313655853271484 nb_pixel_total : 10489 time to create 1 rle with old method : 0.01899552345275879 length of segment : 136 time for calcul the mask position with numpy : 0.0008647441864013672 nb_pixel_total : 11225 time to create 1 rle with old method : 0.018411874771118164 length of segment : 144 time for calcul the mask position with numpy : 0.005486011505126953 nb_pixel_total : 75003 time to create 1 rle with old method : 0.08549189567565918 length of segment : 481 time for calcul the mask position with numpy : 0.0016884803771972656 nb_pixel_total : 27837 time to create 1 rle with old method : 0.03861427307128906 length of segment : 202 time for calcul the mask position with numpy : 0.0030126571655273438 nb_pixel_total : 58348 time to create 1 rle with old method : 0.06490039825439453 length of segment : 312 time for calcul the mask position with numpy : 0.0010056495666503906 nb_pixel_total : 12053 time to create 1 rle with old method : 0.013662576675415039 length of segment : 137 time for calcul the mask position with numpy : 0.0035028457641601562 nb_pixel_total : 85952 time to create 1 rle with old method : 0.09206724166870117 length of segment : 402 time for calcul the mask position with numpy : 0.0009615421295166016 nb_pixel_total : 16040 time to create 1 rle with old method : 0.018943071365356445 length of segment : 153 time for calcul the mask position with numpy : 0.004883766174316406 nb_pixel_total : 90257 time to create 1 rle with old method : 0.10625123977661133 length of segment : 322 time for calcul the mask position with numpy : 0.001800537109375 nb_pixel_total : 27308 time to create 1 rle with old method : 0.04041123390197754 length of segment : 358 time for calcul the mask position with numpy : 0.0010900497436523438 nb_pixel_total : 27670 time to create 1 rle with old method : 0.03653717041015625 length of segment : 192 time for calcul the mask position with numpy : 0.012291908264160156 nb_pixel_total : 185389 time to create 1 rle with new method : 0.04362750053405762 length of segment : 801 time for calcul the mask position with numpy : 0.002024412155151367 nb_pixel_total : 44446 time to create 1 rle with old method : 0.05321502685546875 length of segment : 207 time for calcul the mask position with numpy : 0.0017237663269042969 nb_pixel_total : 37459 time to create 1 rle with old method : 0.050064802169799805 length of segment : 206 time for calcul the mask position with numpy : 0.0020225048065185547 nb_pixel_total : 39709 time to create 1 rle with old method : 0.0479123592376709 length of segment : 201 time for calcul the mask position with numpy : 0.0016741752624511719 nb_pixel_total : 38513 time to create 1 rle with old method : 0.04587817192077637 length of segment : 205 time for calcul the mask position with numpy : 0.0008161067962646484 nb_pixel_total : 14198 time to create 1 rle with old method : 0.016010522842407227 length of segment : 179 time spent for convertir_results : 12.534122705459595 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 132 chid ids of type : 3594 Number RLEs to save : 37485 save missing photos in datou_result : time spend for datou_step_exec : 67.64719128608704 time spend to save output : 2.3085944652557373 total time spend for step 1 : 69.95578575134277 step2:crop_condition Tue Feb 11 21:41:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 132 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 110 About to insert : list_path_to_insert length 110 new photo from crops ! About to upload 110 photos upload in portfolio : 3736932 init cache_photo without model_param we have 110 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739306525_127184 we have uploaded 110 photos in the portfolio 3736932 time of upload the photos Elapsed time : 30.55400061607361 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739306561_127184 we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.91827654838562 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739306566_127184 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5486109256744385 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739306571_127184 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.6639854907989502 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336906282, 1336906277, 1336906272, 1336906183, 1336905872, 1336905794, 1336905762, 1336905753] Looping around the photos to save general results len do output : 132 /1336913093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913137Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913138Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913139Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913140Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336913257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906282', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906277', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906272', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906183', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905872', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905794', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905762', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905753', None, None, None, None, None, '2577803') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 404 time used for this insertion : 0.411754846572876 save_final save missing photos in datou_result : time spend for datou_step_exec : 73.1684799194336 time spend to save output : 0.41602301597595215 total time spend for step 2 : 73.58450293540955 step3:rle_unique_nms_with_priority Tue Feb 11 21:42:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 132 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 23 nb_hashtags : 4 time to prepare the origin masks : 6.6065874099731445 time for calcul the mask position with numpy : 0.5135481357574463 nb_pixel_total : 7387762 time to create 1 rle with new method : 0.779944658279419 time for calcul the mask position with numpy : 0.05945396423339844 nb_pixel_total : 13036 time to create 1 rle with old method : 0.015890836715698242 time for calcul the mask position with numpy : 0.05787372589111328 nb_pixel_total : 25374 time to create 1 rle with old method : 0.03202342987060547 time for calcul the mask position with numpy : 0.05835986137390137 nb_pixel_total : 13084 time to create 1 rle with old method : 0.015227317810058594 time for calcul the mask position with numpy : 0.05571866035461426 nb_pixel_total : 26833 time to create 1 rle with old method : 0.0376436710357666 time for calcul the mask position with numpy : 0.05721855163574219 nb_pixel_total : 13193 time to create 1 rle with old method : 0.015087127685546875 time for calcul the mask position with numpy : 0.05973935127258301 nb_pixel_total : 167652 time to create 1 rle with new method : 0.7132935523986816 time for calcul the mask position with numpy : 0.05016589164733887 nb_pixel_total : 23496 time to create 1 rle with old method : 0.025892257690429688 time for calcul the mask position with numpy : 0.054262638092041016 nb_pixel_total : 52983 time to create 1 rle with old method : 0.06003689765930176 time for calcul the mask position with numpy : 0.056519508361816406 nb_pixel_total : 14572 time to create 1 rle with old method : 0.0229339599609375 time for calcul the mask position with numpy : 0.053043365478515625 nb_pixel_total : 6853 time to create 1 rle with old method : 0.0074977874755859375 time for calcul the mask position with numpy : 0.0527646541595459 nb_pixel_total : 56514 time to create 1 rle with old method : 0.06126809120178223 time for calcul the mask position with numpy : 0.03853344917297363 nb_pixel_total : 11008 time to create 1 rle with old method : 0.012508630752563477 time for calcul the mask position with numpy : 0.04019498825073242 nb_pixel_total : 8584 time to create 1 rle with old method : 0.009288311004638672 time for calcul the mask position with numpy : 0.04023027420043945 nb_pixel_total : 23818 time to create 1 rle with old method : 0.025823116302490234 time for calcul the mask position with numpy : 0.03970980644226074 nb_pixel_total : 21415 time to create 1 rle with old method : 0.023458480834960938 time for calcul the mask position with numpy : 0.029429197311401367 nb_pixel_total : 22601 time to create 1 rle with old method : 0.025485992431640625 time for calcul the mask position with numpy : 0.026604890823364258 nb_pixel_total : 187402 time to create 1 rle with new method : 0.4862520694732666 time for calcul the mask position with numpy : 0.026957988739013672 nb_pixel_total : 33580 time to create 1 rle with old method : 0.03684806823730469 time for calcul the mask position with numpy : 0.02988600730895996 nb_pixel_total : 79287 time to create 1 rle with old method : 0.08290433883666992 time for calcul the mask position with numpy : 0.04133200645446777 nb_pixel_total : 36070 time to create 1 rle with old method : 0.03873872756958008 time for calcul the mask position with numpy : 0.04950308799743652 nb_pixel_total : 11648 time to create 1 rle with old method : 0.013720989227294922 time for calcul the mask position with numpy : 0.04466748237609863 nb_pixel_total : 22129 time to create 1 rle with old method : 0.0238039493560791 time for calcul the mask position with numpy : 0.04835844039916992 nb_pixel_total : 35506 time to create 1 rle with old method : 0.03899669647216797 create new chi : 4.3120338916778564 time to delete rle : 0.024106740951538086 batch 1 Loaded 47 chid ids of type : 3594 +++++++++++++++++++++++++++++++++Number RLEs to save : 12787 TO DO : save crop sub photo not yet done ! save time : 1.3664472103118896 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 7.107072830200195 time for calcul the mask position with numpy : 0.3334486484527588 nb_pixel_total : 7185813 time to create 1 rle with new method : 0.41203951835632324 time for calcul the mask position with numpy : 0.02651691436767578 nb_pixel_total : 12681 time to create 1 rle with old method : 0.018123626708984375 time for calcul the mask position with numpy : 0.029825210571289062 nb_pixel_total : 42210 time to create 1 rle with old method : 0.050004005432128906 time for calcul the mask position with numpy : 0.027467727661132812 nb_pixel_total : 13636 time to create 1 rle with old method : 0.016490936279296875 time for calcul the mask position with numpy : 0.02929997444152832 nb_pixel_total : 120674 time to create 1 rle with old method : 0.13442111015319824 time for calcul the mask position with numpy : 0.026990413665771484 nb_pixel_total : 36992 time to create 1 rle with old method : 0.04562258720397949 time for calcul the mask position with numpy : 0.027915000915527344 nb_pixel_total : 46443 time to create 1 rle with old method : 0.050772905349731445 time for calcul the mask position with numpy : 0.027358293533325195 nb_pixel_total : 48638 time to create 1 rle with old method : 0.052156686782836914 time for calcul the mask position with numpy : 0.025756359100341797 nb_pixel_total : 11509 time to create 1 rle with old method : 0.012974739074707031 time for calcul the mask position with numpy : 0.026430845260620117 nb_pixel_total : 112434 time to create 1 rle with old method : 0.1213529109954834 time for calcul the mask position with numpy : 0.02537989616394043 nb_pixel_total : 35338 time to create 1 rle with old method : 0.037868499755859375 time for calcul the mask position with numpy : 0.029000520706176758 nb_pixel_total : 114298 time to create 1 rle with old method : 0.12229251861572266 time for calcul the mask position with numpy : 0.02682781219482422 nb_pixel_total : 13634 time to create 1 rle with old method : 0.014822244644165039 time for calcul the mask position with numpy : 0.026071786880493164 nb_pixel_total : 127258 time to create 1 rle with old method : 0.20223617553710938 time for calcul the mask position with numpy : 0.025313377380371094 nb_pixel_total : 38105 time to create 1 rle with old method : 0.041784048080444336 time for calcul the mask position with numpy : 0.04477119445800781 nb_pixel_total : 25241 time to create 1 rle with old method : 0.02740955352783203 time for calcul the mask position with numpy : 0.04000258445739746 nb_pixel_total : 87681 time to create 1 rle with old method : 0.09199118614196777 time for calcul the mask position with numpy : 0.04596590995788574 nb_pixel_total : 221815 time to create 1 rle with new method : 1.097266674041748 create new chi : 3.4676077365875244 time to delete rle : 0.0022487640380859375 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 14150 TO DO : save crop sub photo not yet done ! save time : 1.3947668075561523 nb_obj : 16 nb_hashtags : 2 time to prepare the origin masks : 6.507183790206909 time for calcul the mask position with numpy : 0.7778170108795166 nb_pixel_total : 7015474 time to create 1 rle with new method : 0.529085636138916 time for calcul the mask position with numpy : 0.02911090850830078 nb_pixel_total : 47616 time to create 1 rle with old method : 0.08195996284484863 time for calcul the mask position with numpy : 0.03493690490722656 nb_pixel_total : 16360 time to create 1 rle with old method : 0.022772789001464844 time for calcul the mask position with numpy : 0.034218549728393555 nb_pixel_total : 28808 time to create 1 rle with old method : 0.03851008415222168 time for calcul the mask position with numpy : 0.028597116470336914 nb_pixel_total : 33523 time to create 1 rle with old method : 0.03819751739501953 time for calcul the mask position with numpy : 0.027527570724487305 nb_pixel_total : 110026 time to create 1 rle with old method : 0.11899876594543457 time for calcul the mask position with numpy : 0.03025650978088379 nb_pixel_total : 298525 time to create 1 rle with new method : 0.8704543113708496 time for calcul the mask position with numpy : 0.028196334838867188 nb_pixel_total : 12991 time to create 1 rle with old method : 0.014283180236816406 time for calcul the mask position with numpy : 0.026505708694458008 nb_pixel_total : 10471 time to create 1 rle with old method : 0.012283086776733398 time for calcul the mask position with numpy : 0.028879404067993164 nb_pixel_total : 75866 time to create 1 rle with old method : 0.08178973197937012 time for calcul the mask position with numpy : 0.02710580825805664 nb_pixel_total : 51322 time to create 1 rle with old method : 0.061235666275024414 time for calcul the mask position with numpy : 0.028618812561035156 nb_pixel_total : 71501 time to create 1 rle with old method : 0.0776207447052002 time for calcul the mask position with numpy : 0.03001999855041504 nb_pixel_total : 27116 time to create 1 rle with old method : 0.029169082641601562 time for calcul the mask position with numpy : 0.030320405960083008 nb_pixel_total : 186331 time to create 1 rle with new method : 1.0774867534637451 time for calcul the mask position with numpy : 0.030843496322631836 nb_pixel_total : 57650 time to create 1 rle with old method : 0.062412261962890625 time for calcul the mask position with numpy : 0.02805948257446289 nb_pixel_total : 213252 time to create 1 rle with new method : 0.7546617984771729 time for calcul the mask position with numpy : 0.026099443435668945 nb_pixel_total : 37568 time to create 1 rle with old method : 0.0419769287109375 create new chi : 5.298173666000366 time to delete rle : 0.002591848373413086 batch 1 Loaded 33 chid ids of type : 3594 +++++++++++++++++++++Number RLEs to save : 15150 TO DO : save crop sub photo not yet done ! save time : 0.9747765064239502 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 4.218966484069824 time for calcul the mask position with numpy : 0.855536699295044 nb_pixel_total : 7893927 time to create 1 rle with new method : 0.5420300960540771 time for calcul the mask position with numpy : 0.02522730827331543 nb_pixel_total : 12323 time to create 1 rle with old method : 0.013237237930297852 time for calcul the mask position with numpy : 0.026433706283569336 nb_pixel_total : 10310 time to create 1 rle with old method : 0.01147150993347168 time for calcul the mask position with numpy : 0.025252103805541992 nb_pixel_total : 11010 time to create 1 rle with old method : 0.01224517822265625 time for calcul the mask position with numpy : 0.02642822265625 nb_pixel_total : 45237 time to create 1 rle with old method : 0.048545122146606445 time for calcul the mask position with numpy : 0.025992631912231445 nb_pixel_total : 13822 time to create 1 rle with old method : 0.014711618423461914 time for calcul the mask position with numpy : 0.024728059768676758 nb_pixel_total : 65147 time to create 1 rle with old method : 0.07193493843078613 time for calcul the mask position with numpy : 0.025792360305786133 nb_pixel_total : 70131 time to create 1 rle with old method : 0.07198476791381836 time for calcul the mask position with numpy : 0.023854732513427734 nb_pixel_total : 75210 time to create 1 rle with old method : 0.07677626609802246 time for calcul the mask position with numpy : 0.02337026596069336 nb_pixel_total : 19614 time to create 1 rle with old method : 0.020737886428833008 time for calcul the mask position with numpy : 0.024948596954345703 nb_pixel_total : 77669 time to create 1 rle with old method : 0.08091878890991211 create new chi : 2.1194827556610107 time to delete rle : 0.0010938644409179688 batch 1 Loaded 21 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 7286 TO DO : save crop sub photo not yet done ! save time : 0.44939470291137695 nb_obj : 21 nb_hashtags : 2 time to prepare the origin masks : 7.709039211273193 time for calcul the mask position with numpy : 0.3895387649536133 nb_pixel_total : 7559477 time to create 1 rle with new method : 0.6041522026062012 time for calcul the mask position with numpy : 0.027272701263427734 nb_pixel_total : 35760 time to create 1 rle with old method : 0.03940892219543457 time for calcul the mask position with numpy : 0.030477285385131836 nb_pixel_total : 20334 time to create 1 rle with old method : 0.022243738174438477 time for calcul the mask position with numpy : 0.040894269943237305 nb_pixel_total : 27197 time to create 1 rle with old method : 0.029413223266601562 time for calcul the mask position with numpy : 0.04279065132141113 nb_pixel_total : 72045 time to create 1 rle with old method : 0.07728433609008789 time for calcul the mask position with numpy : 0.04175066947937012 nb_pixel_total : 25416 time to create 1 rle with old method : 0.02815842628479004 time for calcul the mask position with numpy : 0.041795969009399414 nb_pixel_total : 21854 time to create 1 rle with old method : 0.023510456085205078 time for calcul the mask position with numpy : 0.03994894027709961 nb_pixel_total : 5815 time to create 1 rle with old method : 0.006940126419067383 time for calcul the mask position with numpy : 0.05068159103393555 nb_pixel_total : 24152 time to create 1 rle with old method : 0.02766108512878418 time for calcul the mask position with numpy : 0.04656505584716797 nb_pixel_total : 38203 time to create 1 rle with old method : 0.044708967208862305 time for calcul the mask position with numpy : 0.05292773246765137 nb_pixel_total : 33275 time to create 1 rle with old method : 0.046346187591552734 time for calcul the mask position with numpy : 0.045630693435668945 nb_pixel_total : 49155 time to create 1 rle with old method : 0.06260299682617188 time for calcul the mask position with numpy : 0.025400161743164062 nb_pixel_total : 24033 time to create 1 rle with old method : 0.026860952377319336 time for calcul the mask position with numpy : 0.025872230529785156 nb_pixel_total : 22440 time to create 1 rle with old method : 0.0252685546875 time for calcul the mask position with numpy : 0.025539636611938477 nb_pixel_total : 44102 time to create 1 rle with old method : 0.04794764518737793 time for calcul the mask position with numpy : 0.027559995651245117 nb_pixel_total : 37493 time to create 1 rle with old method : 0.04367828369140625 time for calcul the mask position with numpy : 0.0316929817199707 nb_pixel_total : 49160 time to create 1 rle with old method : 0.05354571342468262 time for calcul the mask position with numpy : 0.026926279067993164 nb_pixel_total : 47820 time to create 1 rle with old method : 0.052109479904174805 time for calcul the mask position with numpy : 0.02698063850402832 nb_pixel_total : 36647 time to create 1 rle with old method : 0.03900265693664551 time for calcul the mask position with numpy : 0.025438308715820312 nb_pixel_total : 5094 time to create 1 rle with old method : 0.005789041519165039 time for calcul the mask position with numpy : 0.025519847869873047 nb_pixel_total : 79797 time to create 1 rle with old method : 0.09241199493408203 time for calcul the mask position with numpy : 0.02622056007385254 nb_pixel_total : 35131 time to create 1 rle with old method : 0.03853106498718262 create new chi : 2.753220796585083 time to delete rle : 0.0022928714752197266 batch 1 Loaded 43 chid ids of type : 3594 +++++++++++++++++++++++++++++Number RLEs to save : 12830 TO DO : save crop sub photo not yet done ! save time : 0.7585864067077637 nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 7.265063047409058 time for calcul the mask position with numpy : 0.6710193157196045 nb_pixel_total : 7659037 time to create 1 rle with new method : 0.960167646408081 time for calcul the mask position with numpy : 0.04269099235534668 nb_pixel_total : 39141 time to create 1 rle with old method : 0.045105934143066406 time for calcul the mask position with numpy : 0.03874397277832031 nb_pixel_total : 28514 time to create 1 rle with old method : 0.03339076042175293 time for calcul the mask position with numpy : 0.03656125068664551 nb_pixel_total : 51462 time to create 1 rle with old method : 0.055188655853271484 time for calcul the mask position with numpy : 0.03866910934448242 nb_pixel_total : 32680 time to create 1 rle with old method : 0.03793787956237793 time for calcul the mask position with numpy : 0.036727190017700195 nb_pixel_total : 4011 time to create 1 rle with old method : 0.005636692047119141 time for calcul the mask position with numpy : 0.04111504554748535 nb_pixel_total : 33901 time to create 1 rle with old method : 0.03688859939575195 time for calcul the mask position with numpy : 0.038089752197265625 nb_pixel_total : 32504 time to create 1 rle with old method : 0.03650021553039551 time for calcul the mask position with numpy : 0.03611874580383301 nb_pixel_total : 7577 time to create 1 rle with old method : 0.008611440658569336 time for calcul the mask position with numpy : 0.034053802490234375 nb_pixel_total : 57622 time to create 1 rle with old method : 0.0638282299041748 time for calcul the mask position with numpy : 0.0307161808013916 nb_pixel_total : 18493 time to create 1 rle with old method : 0.019939184188842773 time for calcul the mask position with numpy : 0.02495408058166504 nb_pixel_total : 75434 time to create 1 rle with old method : 0.08548974990844727 time for calcul the mask position with numpy : 0.02708888053894043 nb_pixel_total : 37661 time to create 1 rle with old method : 0.04154777526855469 time for calcul the mask position with numpy : 0.031276702880859375 nb_pixel_total : 17072 time to create 1 rle with old method : 0.023274660110473633 time for calcul the mask position with numpy : 0.05064702033996582 nb_pixel_total : 46168 time to create 1 rle with old method : 0.07358741760253906 time for calcul the mask position with numpy : 0.04357481002807617 nb_pixel_total : 16936 time to create 1 rle with old method : 0.018439531326293945 time for calcul the mask position with numpy : 0.04076743125915527 nb_pixel_total : 13785 time to create 1 rle with old method : 0.015621662139892578 time for calcul the mask position with numpy : 0.04467964172363281 nb_pixel_total : 43615 time to create 1 rle with old method : 0.0603785514831543 time for calcul the mask position with numpy : 0.04076075553894043 nb_pixel_total : 32366 time to create 1 rle with old method : 0.0355072021484375 time for calcul the mask position with numpy : 0.040454864501953125 nb_pixel_total : 24656 time to create 1 rle with old method : 0.02650284767150879 time for calcul the mask position with numpy : 0.04493141174316406 nb_pixel_total : 21765 time to create 1 rle with old method : 0.028831958770751953 create new chi : 3.199144124984741 time to delete rle : 0.003749847412109375 batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 11486 TO DO : save crop sub photo not yet done ! save time : 0.6861319541931152 nb_obj : 11 nb_hashtags : 1 time to prepare the origin masks : 4.051246166229248 time for calcul the mask position with numpy : 0.588864803314209 nb_pixel_total : 7938492 time to create 1 rle with new method : 0.9116871356964111 time for calcul the mask position with numpy : 0.025040864944458008 nb_pixel_total : 75003 time to create 1 rle with old method : 0.08039689064025879 time for calcul the mask position with numpy : 0.028098106384277344 nb_pixel_total : 10529 time to create 1 rle with old method : 0.016947031021118164 time for calcul the mask position with numpy : 0.027379512786865234 nb_pixel_total : 10489 time to create 1 rle with old method : 0.012726306915283203 time for calcul the mask position with numpy : 0.025687456130981445 nb_pixel_total : 96023 time to create 1 rle with old method : 0.10240912437438965 time for calcul the mask position with numpy : 0.02830338478088379 nb_pixel_total : 4847 time to create 1 rle with old method : 0.006206512451171875 time for calcul the mask position with numpy : 0.028815746307373047 nb_pixel_total : 27427 time to create 1 rle with old method : 0.03419947624206543 time for calcul the mask position with numpy : 0.030388593673706055 nb_pixel_total : 37929 time to create 1 rle with old method : 0.044901371002197266 time for calcul the mask position with numpy : 0.028818845748901367 nb_pixel_total : 10520 time to create 1 rle with old method : 0.012701272964477539 time for calcul the mask position with numpy : 0.028139114379882812 nb_pixel_total : 27699 time to create 1 rle with old method : 0.03648805618286133 time for calcul the mask position with numpy : 0.030719280242919922 nb_pixel_total : 21131 time to create 1 rle with old method : 0.02793598175048828 time for calcul the mask position with numpy : 0.04026222229003906 nb_pixel_total : 34311 time to create 1 rle with old method : 0.03738117218017578 create new chi : 2.282122850418091 time to delete rle : 0.0011754035949707031 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 7680 TO DO : save crop sub photo not yet done ! save time : 0.48347949981689453 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 8.025789737701416 time for calcul the mask position with numpy : 0.7715277671813965 nb_pixel_total : 7590520 time to create 1 rle with new method : 1.0660746097564697 time for calcul the mask position with numpy : 0.03602433204650879 nb_pixel_total : 14198 time to create 1 rle with old method : 0.019354581832885742 time for calcul the mask position with numpy : 0.030904531478881836 nb_pixel_total : 38513 time to create 1 rle with old method : 0.046738386154174805 time for calcul the mask position with numpy : 0.03078460693359375 nb_pixel_total : 39709 time to create 1 rle with old method : 0.045351505279541016 time for calcul the mask position with numpy : 0.026134252548217773 nb_pixel_total : 37459 time to create 1 rle with old method : 0.04388618469238281 time for calcul the mask position with numpy : 0.025324344635009766 nb_pixel_total : 43147 time to create 1 rle with old method : 0.049916982650756836 time for calcul the mask position with numpy : 0.027322769165039062 nb_pixel_total : 185389 time to create 1 rle with new method : 0.8759486675262451 time for calcul the mask position with numpy : 0.027727127075195312 nb_pixel_total : 27670 time to create 1 rle with old method : 0.03501296043395996 time for calcul the mask position with numpy : 0.02522897720336914 nb_pixel_total : 27308 time to create 1 rle with old method : 0.031504154205322266 time for calcul the mask position with numpy : 0.02781844139099121 nb_pixel_total : 90257 time to create 1 rle with old method : 0.11200737953186035 time for calcul the mask position with numpy : 0.02518916130065918 nb_pixel_total : 16040 time to create 1 rle with old method : 0.01741957664489746 time for calcul the mask position with numpy : 0.02608346939086914 nb_pixel_total : 85952 time to create 1 rle with old method : 0.09209513664245605 time for calcul the mask position with numpy : 0.024816513061523438 nb_pixel_total : 12053 time to create 1 rle with old method : 0.013673543930053711 time for calcul the mask position with numpy : 0.025060653686523438 nb_pixel_total : 58348 time to create 1 rle with old method : 0.06258678436279297 time for calcul the mask position with numpy : 0.026611804962158203 nb_pixel_total : 27837 time to create 1 rle with old method : 0.030545711517333984 create new chi : 3.9734013080596924 time to delete rle : 0.0014829635620117188 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 9829 TO DO : save crop sub photo not yet done ! save time : 0.7897369861602783 map_output_result : {1336906282: (0.0, 'Should be the crop_list due to order', 0.0), 1336906277: (0.0, 'Should be the crop_list due to order', 0.0), 1336906272: (0.0, 'Should be the crop_list due to order', 0.0), 1336906183: (0.0, 'Should be the crop_list due to order', 0.0), 1336905872: (0.0, 'Should be the crop_list due to order', 0.0), 1336905794: (0.0, 'Should be the crop_list due to order', 0.0), 1336905762: (0.0, 'Should be the crop_list due to order', 0.0), 1336905753: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1336906282, 1336906277, 1336906272, 1336906183, 1336905872, 1336905794, 1336905762, 1336905753] Looping around the photos to save general results len do output : 8 /1336906282.Didn't retrieve data . /1336906277.Didn't retrieve data . /1336906272.Didn't retrieve data . /1336906183.Didn't retrieve data . /1336905872.Didn't retrieve data . /1336905794.Didn't retrieve data . /1336905762.Didn't retrieve data . /1336905753.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906282', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906277', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906272', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906183', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905872', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905794', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905762', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905753', None, None, None, None, None, '2577803') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.013518095016479492 save_final save missing photos in datou_result : time spend for datou_step_exec : 87.07005500793457 time spend to save output : 0.017390012741088867 total time spend for step 3 : 87.08744502067566 step4:ventilate_hashtags_in_portfolio Tue Feb 11 21:44:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20462119 get user id for portfolio 20462119 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462119 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','metal','pehd','mal_croppe','papier','pet_fonce','environnement','background','autre','carton','pet_clair')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462119 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','metal','pehd','mal_croppe','papier','pet_fonce','environnement','background','autre','carton','pet_clair')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462119 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','metal','pehd','mal_croppe','papier','pet_fonce','environnement','background','autre','carton','pet_clair')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20462264,20462265,20462266,20462267,20462268,20462269,20462270,20462271,20462272,20462273,20462274?tags=flou,metal,pehd,mal_croppe,papier,pet_fonce,environnement,background,autre,carton,pet_clair Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1336906282, 1336906277, 1336906272, 1336906183, 1336905872, 1336905794, 1336905762, 1336905753] Looping around the photos to save general results len do output : 1 /20462119. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906282', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906277', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906272', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906183', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905872', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905794', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905762', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905753', None, None, None, None, None, '2577803') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.01274561882019043 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.7240159511566162 time spend to save output : 0.012969017028808594 total time spend for step 4 : 1.7369849681854248 step5:final Tue Feb 11 21:44:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1336906282: ('0.0923017035590278',), 1336906277: ('0.0923017035590278',), 1336906272: ('0.0923017035590278',), 1336906183: ('0.0923017035590278',), 1336905872: ('0.0923017035590278',), 1336905794: ('0.0923017035590278',), 1336905762: ('0.0923017035590278',), 1336905753: ('0.0923017035590278',)} new output for save of step final : {1336906282: ('0.0923017035590278',), 1336906277: ('0.0923017035590278',), 1336906272: ('0.0923017035590278',), 1336906183: ('0.0923017035590278',), 1336905872: ('0.0923017035590278',), 1336905794: ('0.0923017035590278',), 1336905762: ('0.0923017035590278',), 1336905753: ('0.0923017035590278',)} [1336906282, 1336906277, 1336906272, 1336906183, 1336905872, 1336905794, 1336905762, 1336905753] Looping around the photos to save general results len do output : 8 /1336906282.Didn't retrieve data . /1336906277.Didn't retrieve data . /1336906272.Didn't retrieve data . /1336906183.Didn't retrieve data . /1336905872.Didn't retrieve data . /1336905794.Didn't retrieve data . /1336905762.Didn't retrieve data . /1336905753.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906282', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906277', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906272', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906183', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905872', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905794', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905762', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905753', None, None, None, None, None, '2577803') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.013845443725585938 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13098883628845215 time spend to save output : 0.014487981796264648 total time spend for step 5 : 0.1454768180847168 step6:blur_detection Tue Feb 11 21:44:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959.jpg resize: (2160, 3840) 1336906282 -5.052978950776224 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec.jpg resize: (2160, 3840) 1336906277 -6.041653814075307 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c.jpg resize: (2160, 3840) 1336906272 -6.160229971952312 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76.jpg resize: (2160, 3840) 1336906183 -5.642515165747087 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937.jpg resize: (2160, 3840) 1336905872 -6.0755037294721355 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02.jpg resize: (2160, 3840) 1336905794 -5.570362950953068 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34.jpg resize: (2160, 3840) 1336905762 -5.548158727344337 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73.jpg resize: (2160, 3840) 1336905753 -5.13135399179276 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646117_0.png resize: (144, 168) 1336913093 -3.4443058273626197 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646112_0.png resize: (334, 252) 1336913094 -2.520433348833472 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646108_0.png resize: (106, 121) 1336913097 -3.5282019006595933 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646106_0.png resize: (256, 147) 1336913101 -2.8594072612366968 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646110_0.png resize: (77, 117) 1336913105 -1.1868563568104138 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646109_0.png resize: (335, 235) 1336913108 -3.940293811878814 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646099_0.png resize: (122, 158) 1336913109 -3.3440174986755276 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646107_0.png resize: (146, 76) 1336913110 -2.7583509093143768 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646098_0.png resize: (234, 194) 1336913111 -2.357611043780161 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646097_0.png resize: (219, 222) 1336913112 -2.7932214186959716 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646102_0.png resize: (211, 207) 1336913113 -2.7676167340326923 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646103_0.png resize: (533, 537) 1336913114 -2.4112318322579407 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646104_0.png resize: (180, 186) 1336913115 -2.4780157905579543 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646111_0.png resize: (152, 150) 1336913116 -3.5669075038571925 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646119_0.png resize: (112, 175) 1336913117 -4.6687889097321165 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646100_0.png resize: (195, 250) 1336913118 -3.2062334201796614 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646118_0.png resize: (197, 167) 1336913119 -3.632569807257411 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646101_0.png resize: (377, 343) 1336913120 -2.723367495431985 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646126_0.png resize: (519, 629) 1336913121 -3.694899198455751 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646134_0.png resize: (140, 150) 1336913122 -3.425870719932745 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646135_0.png resize: (323, 192) 1336913123 -3.1251741844017658 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646120_0.png resize: (847, 358) 1336913124 -2.342828242419922 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646131_0.png resize: (423, 176) 1336913125 -3.0000944124066007 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646128_0.png resize: (522, 360) 1336913126 -3.3406120375847097 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646122_0.png resize: (257, 171) 1336913127 -3.5541031261732003 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646132_0.png resize: (343, 332) 1336913128 -2.754751417240146 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646130_0.png resize: (222, 338) 1336913129 -4.20655218141152 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646121_0.png resize: (399, 403) 1336913130 -3.547868605490165 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646127_0.png resize: (343, 149) 1336913131 -4.204422904895291 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646136_0.png resize: (189, 166) 1336913132 -3.893187352654363 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646129_0.png resize: (159, 106) 1336913133 -3.9378083922197864 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646125_0.png resize: (136, 149) 1336913134 -3.416168517807649 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646141_0.png resize: (410, 144) 1336913135 -3.9390348103437587 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646137_0.png resize: (259, 254) 1336913136 -3.7784154317611236 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646143_0.png resize: (432, 193) 1336913137 -3.10160142704475 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646152_0.png resize: (355, 215) 1336913138 -3.5238153320600696 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646138_0.png resize: (843, 366) 1336913139 -2.727046064505629 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646145_0.png resize: (152, 83) 1336913140 1.0640004910067933 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646142_0.png resize: (417, 253) 1336913141 -3.9568467128992433 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646140_0.png resize: (611, 454) 1336913142 -4.0870066405770284 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646146_0.png resize: (142, 113) 1336913143 -0.7278026593774373 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646151_0.png resize: (193, 194) 1336913144 -3.3957824038230715 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646148_0.png resize: (741, 372) 1336913145 -4.577662346680568 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646139_0.png resize: (244, 393) 1336913146 -4.3030127826788895 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646144_0.png resize: (358, 383) 1336913147 -3.816786186511252 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646149_0.png resize: (328, 145) 1336913148 -4.504445201291844 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646155_0.png resize: (341, 279) 1336913149 -2.929298318341537 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646159_0.png resize: (297, 208) 1336913150 -3.331758383915098 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646162_0.png resize: (157, 150) 1336913151 -3.5033449396256904 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646153_0.png resize: (351, 363) 1336913152 -3.6718681902870873 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646160_0.png resize: (101, 154) 1336913153 -2.795393631222449 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646157_0.png resize: (256, 376) 1336913158 -4.164493915620266 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646154_0.png resize: (163, 268) 1336913161 -3.4054304682947287 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646178_0.png resize: (160, 163) 1336913162 -2.19832570918612 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646182_0.png resize: (104, 274) 1336913163 -3.63393118072172 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646181_0.png resize: (228, 184) 1336913164 -3.299948102918949 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646163_0.png resize: (208, 275) 1336913165 -0.002111665692981294 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646169_0.png resize: (306, 190) 1336913166 -2.596493258882124 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646165_0.png resize: (77, 110) 1336913167 -2.8911976067840968 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646164_0.png resize: (312, 349) 1336913168 -2.7789036113244125 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646174_0.png resize: (288, 202) 1336913169 -4.08778689107609 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646172_0.png resize: (178, 203) 1336913170 -3.703663615084956 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646170_0.png resize: (340, 265) 1336913171 -2.0548243037286293 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646173_0.png resize: (408, 325) 1336913172 -1.8177924183471745 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646175_0.png resize: (225, 234) 1336913173 -4.409258902532535 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646179_0.png resize: (157, 225) 1336913174 -2.419176973969477 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646176_0.png resize: (202, 182) 1336913175 -2.424203382779717 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646171_0.png resize: (354, 160) 1336913176 -2.7164724212870985 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646168_0.png resize: (232, 287) 1336913177 -1.097831434104717 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646166_0.png resize: (206, 265) 1336913178 -4.489524564622447 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646177_0.png resize: (66, 125) 1336913179 -1.4394470453329506 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646167_0.png resize: (352, 238) 1336913180 -1.7396605456618932 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646184_0.png resize: (128, 245) 1336913181 -3.261591497331408 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646190_0.png resize: (256, 298) 1336913182 -4.4525709147684935 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646185_0.png resize: (230, 165) 1336913183 -3.1476860603096264 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646187_0.png resize: (309, 237) 1336913184 -3.1492384985024082 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646188_0.png resize: (140, 184) 1336913185 -3.166412736608207 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646203_0.png resize: (285, 155) 1336913186 -3.205432730239953 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646200_0.png resize: (272, 205) 1336913187 -4.198155641825421 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646202_0.png resize: (167, 236) 1336913188 -3.4126444846125157 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646186_0.png resize: (207, 297) 1336913189 -3.611681188971325 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646199_0.png resize: (258, 173) 1336913190 -3.0542206614453686 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646191_0.png resize: (124, 172) 1336913191 -3.488462430381974 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646193_0.png resize: (350, 330) 1336913192 -3.471577444206366 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646194_0.png resize: (196, 131) 1336913193 -2.962110199493002 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646195_0.png resize: (341, 273) 1336913194 -4.315864887711979 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646192_0.png resize: (287, 216) 1336913195 -4.106105302636302 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646189_0.png resize: (119, 208) 1336913196 -4.325732825145126 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646196_0.png resize: (121, 81) 1336913197 -3.087628718738093 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646197_0.png resize: (240, 188) 1336913198 -4.146300043614879 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646211_0.png resize: (506, 261) 1336913199 -3.2486668192552957 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646205_0.png resize: (196, 221) 1336913200 -2.6571718678749523 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646206_0.png resize: (188, 182) 1336913201 -4.227776021976225 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646214_0.png resize: (349, 383) 1336913202 -3.527725520248422 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646212_0.png resize: (136, 130) 1336913203 -3.877512687452687 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646204_0.png resize: (252, 235) 1336913204 -3.891329489429293 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646213_0.png resize: (144, 100) 1336913206 -3.151318055650228 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646208_0.png resize: (212, 368) 1336913207 -3.2680694609177117 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646210_0.png resize: (98, 82) 1336913208 -2.757374919499104 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646207_0.png resize: (135, 108) 1336913209 -2.087049594588111 treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646209_0.png resize: (290, 138) 1336913210 -3.335768441510481 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646225_0.png resize: (205, 245) 1336913211 -4.520837427685498 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646219_0.png resize: (153, 125) 1336913212 -4.330941320069301 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646216_0.png resize: (312, 360) 1336913213 -3.440024409044096 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646224_0.png resize: (204, 329) 1336913214 -3.647694276307913 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646223_0.png resize: (605, 775) 1336913215 -3.8219340472004273 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646215_0.png resize: (202, 196) 1336913216 -4.052726081075318 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646221_0.png resize: (228, 182) 1336913217 -3.8988871023045646 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646217_0.png resize: (137, 171) 1336913218 -4.083533336475305 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646222_0.png resize: (191, 155) 1336913219 -3.616492437647174 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646113_0.png resize: (264, 128) 1336913228 -2.787051375382404 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646105_0.png resize: (178, 173) 1336913229 -2.939789114733766 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646115_0.png resize: (119, 200) 1336913230 -2.419578310688703 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646123_0.png resize: (239, 262) 1336913231 -2.949711143716184 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646150_0.png resize: (173, 252) 1336913232 -2.275806606121061 treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646147_0.png resize: (560, 676) 1336913233 -4.505842640499994 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646161_0.png resize: (109, 132) 1336913234 -1.365884257007446 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646158_0.png resize: (140, 159) 1336913235 -3.291883153283807 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646183_0.png resize: (206, 220) 1336913236 -3.136007826267733 treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646180_0.png resize: (360, 290) 1336913238 -3.8339740167497065 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646198_0.png resize: (256, 202) 1336913239 -4.573652904873071 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646218_0.png resize: (399, 290) 1336913240 -4.1170017103790535 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646226_0.png resize: (182, 337) 1336913241 -4.381423024477184 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646227_0.png resize: (178, 356) 1336913242 -2.901685931010898 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646228_0.png resize: (179, 134) 1336913244 -3.6980185202312046 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646116_0.png resize: (225, 173) 1336913247 -3.9097959033733747 treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646114_0.png resize: (362, 715) 1336913252 -2.519098506775381 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646124_0.png resize: (288, 559) 1336913253 -3.880015439408048 treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646133_0.png resize: (296, 502) 1336913254 -4.5779151599556345 treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646156_0.png resize: (292, 322) 1336913255 -2.909457586978823 treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646201_0.png resize: (272, 277) 1336913256 -4.390722330563489 treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646220_0.png resize: (307, 384) 1336913257 -4.052348950681741 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 140 time used for this insertion : 0.019450664520263672 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 140 time used for this insertion : 0.030727148056030273 save missing photos in datou_result : time spend for datou_step_exec : 31.144275665283203 time spend to save output : 0.05705666542053223 total time spend for step 6 : 31.201332330703735 step7:brightness Tue Feb 11 21:44:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959.jpg treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec.jpg treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c.jpg treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76.jpg treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937.jpg treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02.jpg treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34.jpg treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73.jpg treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646117_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646112_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646108_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646106_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646110_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646109_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646099_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646107_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646098_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646097_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646102_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646103_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646104_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646111_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646119_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646100_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646118_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646101_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646126_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646134_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646135_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646120_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646131_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646128_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646122_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646132_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646130_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646121_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646127_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646136_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646129_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646125_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646141_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646137_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646143_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646152_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646138_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646145_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646142_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646140_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646146_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646151_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646148_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646139_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646144_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646149_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646155_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646159_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646162_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646153_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646160_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646157_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646154_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646178_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646182_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646181_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646163_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646169_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646165_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646164_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646174_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646172_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646170_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646173_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646175_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646179_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646176_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646171_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646168_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646166_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646177_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646167_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646184_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646190_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646185_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646187_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646188_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646203_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646200_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646202_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646186_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646199_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646191_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646193_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646194_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646195_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646192_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646189_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646196_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646197_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646211_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646205_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646206_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646214_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646212_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646204_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646213_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646208_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646210_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646207_0.png treat image : temp/1739306427_127184_1336905762_46c61059ad9b797e3d1f0907bc0fce34_rle_crop_3670646209_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646225_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646219_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646216_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646224_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646223_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646215_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646221_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646217_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646222_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646113_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646105_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646115_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646123_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646150_0.png treat image : temp/1739306427_127184_1336906272_d07af4f413c3bd6fd58ea6c38bf0357c_rle_crop_3670646147_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646161_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646158_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646183_0.png treat image : temp/1739306427_127184_1336905872_3e4ccf78bc4278057574ea54540c5937_rle_crop_3670646180_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646198_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646218_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646226_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646227_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646228_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646116_0.png treat image : temp/1739306427_127184_1336906282_2ff3206dd3793b726b22de25d9f2c959_rle_crop_3670646114_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646124_0.png treat image : temp/1739306427_127184_1336906277_09af41293510982c63b6f7e515fb95ec_rle_crop_3670646133_0.png treat image : temp/1739306427_127184_1336906183_20ecb48ad40d72ee9a6bd16caaac5a76_rle_crop_3670646156_0.png treat image : temp/1739306427_127184_1336905794_8cc0a2df3799681ca0c407dde7dd3c02_rle_crop_3670646201_0.png treat image : temp/1739306427_127184_1336905753_1e4d6bcd0ad6817c6bf16abe061c1f73_rle_crop_3670646220_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 140 time used for this insertion : 0.5718567371368408 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 140 time used for this insertion : 0.02931499481201172 save missing photos in datou_result : time spend for datou_step_exec : 8.796619176864624 time spend to save output : 0.6063580513000488 total time spend for step 7 : 9.402977228164673 step8:velours_tree Tue Feb 11 21:45:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.4515807628631592 time spend to save output : 5.984306335449219e-05 total time spend for step 8 : 0.45164060592651367 step9:send_mail_cod Tue Feb 11 21:45:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20462119_11-02-2025_21_45_02.pdf 20462264 imagette204622641739306702 20462265 change filename to text .imagette204622651739306702 20462266 imagette204622661739306702 20462267 imagette204622671739306702 20462268 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204622681739306702 20462269 imagette204622691739306704 20462271 imagette204622711739306704 20462272 imagette204622721739306704 20462273 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204622731739306704 20462274 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204622741739306705 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20462119 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20462264,20462265,20462266,20462267,20462268,20462269,20462270,20462271,20462272,20462273,20462274?tags=flou,metal,pehd,mal_croppe,papier,pet_fonce,environnement,background,autre,carton,pet_clair args[1336906282] : ((1336906282, -5.052978950776224, 492609224), (1336906282, 0.0629468877211723, 2107752395), '0.0923017035590278') apple ((1336906282, -5.052978950776224, 492609224), (1336906282, 0.0629468877211723, 2107752395), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336906277] : ((1336906277, -6.041653814075307, 492609224), (1336906277, 0.2446214459005884, 2107752395), '0.0923017035590278') apple ((1336906277, -6.041653814075307, 492609224), (1336906277, 0.2446214459005884, 2107752395), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336906272] : ((1336906272, -6.160229971952312, 492609224), (1336906272, 0.20787051478929924, 2107752395), '0.0923017035590278') apple ((1336906272, -6.160229971952312, 492609224), (1336906272, 0.20787051478929924, 2107752395), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336906183] : ((1336906183, -5.642515165747087, 492609224), (1336906183, 0.2071004048836526, 2107752395), '0.0923017035590278') apple ((1336906183, -5.642515165747087, 492609224), (1336906183, 0.2071004048836526, 2107752395), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336905872] : ((1336905872, -6.0755037294721355, 492609224), (1336905872, 0.43868507986768335, 2107752395), '0.0923017035590278') apple ((1336905872, -6.0755037294721355, 492609224), (1336905872, 0.43868507986768335, 2107752395), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336905794] : ((1336905794, -5.570362950953068, 492609224), (1336905794, -0.5091675511922209, 501862349), '0.0923017035590278') apple ((1336905794, -5.570362950953068, 492609224), (1336905794, -0.5091675511922209, 501862349), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336905762] : ((1336905762, -5.548158727344337, 492609224), (1336905762, -0.07411321573706178, 496442774), '0.0923017035590278') apple ((1336905762, -5.548158727344337, 492609224), (1336905762, -0.07411321573706178, 496442774), '0.0923017035590278') We are sending mail with results at report@fotonower.com args[1336905753] : ((1336905753, -5.13135399179276, 492609224), (1336905753, -0.745766304978792, 501862349), '0.0923017035590278') apple ((1336905753, -5.13135399179276, 492609224), (1336905753, -0.745766304978792, 501862349), '0.0923017035590278') We are sending mail with results at report@fotonower.com refus_total : 0.0923017035590278 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20462119 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462119_11-02-2025_21_45_02.pdf results_Auto_P20462119_11-02-2025_21_45_02.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462119_11-02-2025_21_45_02.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','20462119','results_Auto_P20462119_11-02-2025_21_45_02.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462119_11-02-2025_21_45_02.pdf','pdf','','0.57','0.0923017035590278') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/20462119

https://www.fotonower.com/image?json=false&list_photos_id=1336906282
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336906277
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336906272
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336906183
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336905872
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336905794
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336905762
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1336905753
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 9.23%
Veuillez trouver les photos des contaminants.

exemples de contaminants: metal: https://www.fotonower.com/view/20462265?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/20462268?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/20462273?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/20462274?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462119_11-02-2025_21_45_02.pdf.

Lien vers velours :https://www.fotonower.com/velours/20462264,20462265,20462266,20462267,20462268,20462269,20462270,20462271,20462272,20462273,20462274?tags=flou,metal,pehd,mal_croppe,papier,pet_fonce,environnement,background,autre,carton,pet_clair.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 11 Feb 2025 20:45:09 GMT Content-Length: 0 Connection: close X-Message-Id: nPXDl8TaTmGWH5LHoBQ-qw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1336906282, 1336906277, 1336906272, 1336906183, 1336905872, 1336905794, 1336905762, 1336905753] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906282', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906277', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906272', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906183', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905872', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905794', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905762', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905753', None, None, None, None, None, '2577803') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.014892101287841797 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.956168174743652 time spend to save output : 0.015057563781738281 total time spend for step 9 : 6.971225738525391 step10:split_time_score Tue Feb 11 21:45:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11022025 20462119 Nombre de photos uploadées : 8 / 23040 (0%) 11022025 20462119 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 11022025 20462119 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 4.0531158447265625e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0003554821014404297 elapsed_time : insert_dashboard_record_day_entry 0.02475118637084961 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.07691123425069783 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20431073_11-02-2025_08_36_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20431073 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20431073 AND mptpi.`type`=3726 To do Qualite : 0.10961501886815206 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20431074_11-02-2025_07_32_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20431074 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20431074 AND mptpi.`type`=3594 To do Qualite : 0.02706148847995352 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20436919_11-02-2025_09_59_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20436919 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20436919 AND mptpi.`type`=3726 To do Qualite : 0.04376340975932158 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20436385_11-02-2025_09_10_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20436385 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20436385 AND mptpi.`type`=3726 To do Qualite : 0.09417503197788053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20455314_11-02-2025_18_45_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20455314 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20455314 AND mptpi.`type`=3726 To do Qualite : 0.022027677852884973 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20443833_11-02-2025_12_56_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20443833 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20443833 AND mptpi.`type`=3726 To do Qualite : 0.04597244815394182 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20445953_11-02-2025_15_34_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20445953 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20445953 AND mptpi.`type`=3726 To do Qualite : 0.01567774756924404 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20445957_11-02-2025_15_03_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20445957 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20445957 AND mptpi.`type`=3726 To do Qualite : 0.0410884727923565 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20450252_11-02-2025_17_02_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20450252 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20450252 AND mptpi.`type`=3726 To do Qualite : 0.016970602825565714 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20454228_11-02-2025_18_19_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20454228 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20454228 AND mptpi.`type`=3726 To do Qualite : 0.0993477229292653 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20459162_11-02-2025_20_06_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20459162 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20459162 AND mptpi.`type`=3726 To do Qualite : 0.0923017035590278 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462119_11-02-2025_21_45_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462119 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462119 AND mptpi.`type`=3594 To do Qualite : 0.052604988588719925 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462120_11-02-2025_21_37_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462120 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462120 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11022025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1336906282, 1336906277, 1336906272, 1336906183, 1336905872, 1336905794, 1336905762, 1336905753] Looping around the photos to save general results len do output : 1 /20462119Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906282', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906277', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906272', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336906183', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905872', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905794', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905762', None, None, None, None, None, '2577803') ('3318', None, None, None, None, None, None, None, '2577803') ('3318', '20462119', '1336905753', None, None, None, None, None, '2577803') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.2952754497528076 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.9825875759124756 time spend to save output : 0.2954673767089844 total time spend for step 10 : 1.27805495262146 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 131.49user 105.27system 4:46.82elapsed 82%CPU (0avgtext+0avgdata 5122088maxresident)k 441528inputs+106928outputs (18084major+13277660minor)pagefaults 0swaps