python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 363352 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2843491'] with mtr_portfolio_ids : ['22934121'] and first list_photo_ids : [] new path : /proc/363352/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 1.3862385749816895 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue May 13 22:30:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4332 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-13 22:30:35.284222: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-13 22:30:35.319179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-13 22:30:35.321429: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f6bcc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-13 22:30:35.321478: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-13 22:30:35.326196: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-13 22:30:35.607139: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x33d08be0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-13 22:30:35.607199: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-13 22:30:35.608611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-13 22:30:35.610683: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-13 22:30:35.643922: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-13 22:30:35.663224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-13 22:30:35.667960: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-13 22:30:35.704446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-13 22:30:35.709266: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-13 22:30:35.764957: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-13 22:30:35.766223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-13 22:30:35.766652: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-13 22:30:35.768229: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-13 22:30:35.768249: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-13 22:30:35.768259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-13 22:30:35.769651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3880 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-13 22:30:36.147859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-13 22:30:36.148022: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-13 22:30:36.148055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-13 22:30:36.148084: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-13 22:30:36.148113: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-13 22:30:36.148141: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-13 22:30:36.148167: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-13 22:30:36.148195: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-13 22:30:36.149346: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-13 22:30:36.150925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-13 22:30:36.150979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-13 22:30:36.150996: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-13 22:30:36.151011: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-13 22:30:36.151025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-13 22:30:36.151039: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-13 22:30:36.151052: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-13 22:30:36.151067: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-13 22:30:36.151971: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-13 22:30:36.152009: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-13 22:30:36.152018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-13 22:30:36.152026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-13 22:30:36.152985: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3880 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-13 22:30:47.115143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-13 22:30:47.492428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-13 22:30:49.346381: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:49.347217: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.96G (3180212992 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:49.348003: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.67G (2862191616 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:49.348745: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.40G (2575972352 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.091985: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-13 22:30:50.148745: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.148831: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.149527: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.149547: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.156470: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.156509: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.162718: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.162744: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.163421: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.163439: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.199062: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.199116: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.199127: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-13 22:30:50.200046: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.200076: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.201410: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.201439: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.208849: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.208887: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-13 22:30:50.209926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.211087: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.211957: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.220088: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.220695: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.247886: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.248738: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.249415: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.250058: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.254215: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.254856: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.255522: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.256185: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.258047: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.267537: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.268223: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.277426: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.278104: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.278758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.279423: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.280098: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-13 22:30:50.280723: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.29G (3533570048 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 36 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 10.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 35 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 32 Detection mask done ! Trying to reset tf kernel 364047 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1946 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3139 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.022312641143798828 nb_pixel_total : 428765 time to create 1 rle with new method : 0.05726885795593262 length of segment : 784 time for calcul the mask position with numpy : 0.0009562969207763672 nb_pixel_total : 20575 time to create 1 rle with old method : 0.030389070510864258 length of segment : 141 time for calcul the mask position with numpy : 0.0027556419372558594 nb_pixel_total : 80160 time to create 1 rle with old method : 0.09562516212463379 length of segment : 272 time for calcul the mask position with numpy : 0.002212047576904297 nb_pixel_total : 63038 time to create 1 rle with old method : 0.0780327320098877 length of segment : 237 time for calcul the mask position with numpy : 0.004381656646728516 nb_pixel_total : 129558 time to create 1 rle with old method : 0.15927457809448242 length of segment : 443 time for calcul the mask position with numpy : 0.0019867420196533203 nb_pixel_total : 61923 time to create 1 rle with old method : 0.07772278785705566 length of segment : 274 time for calcul the mask position with numpy : 0.002500772476196289 nb_pixel_total : 80085 time to create 1 rle with old method : 0.09379935264587402 length of segment : 308 time for calcul the mask position with numpy : 0.004727363586425781 nb_pixel_total : 69827 time to create 1 rle with old method : 0.0875089168548584 length of segment : 789 time for calcul the mask position with numpy : 0.003683328628540039 nb_pixel_total : 90600 time to create 1 rle with old method : 0.10517287254333496 length of segment : 264 time for calcul the mask position with numpy : 0.0014672279357910156 nb_pixel_total : 40081 time to create 1 rle with old method : 0.047261953353881836 length of segment : 250 time for calcul the mask position with numpy : 0.0006518363952636719 nb_pixel_total : 18246 time to create 1 rle with old method : 0.02195906639099121 length of segment : 133 time for calcul the mask position with numpy : 0.0023581981658935547 nb_pixel_total : 76415 time to create 1 rle with old method : 0.08687567710876465 length of segment : 352 time for calcul the mask position with numpy : 0.0005855560302734375 nb_pixel_total : 19321 time to create 1 rle with old method : 0.022846221923828125 length of segment : 148 time for calcul the mask position with numpy : 0.0004475116729736328 nb_pixel_total : 10594 time to create 1 rle with old method : 0.012987375259399414 length of segment : 130 time for calcul the mask position with numpy : 0.0009508132934570312 nb_pixel_total : 32037 time to create 1 rle with old method : 0.03973722457885742 length of segment : 233 time for calcul the mask position with numpy : 0.0004246234893798828 nb_pixel_total : 20142 time to create 1 rle with old method : 0.024616479873657227 length of segment : 146 time for calcul the mask position with numpy : 0.0013480186462402344 nb_pixel_total : 44382 time to create 1 rle with old method : 0.05042409896850586 length of segment : 261 time for calcul the mask position with numpy : 0.0007865428924560547 nb_pixel_total : 29386 time to create 1 rle with old method : 0.03444409370422363 length of segment : 206 time for calcul the mask position with numpy : 0.0004076957702636719 nb_pixel_total : 11449 time to create 1 rle with old method : 0.013940572738647461 length of segment : 133 time for calcul the mask position with numpy : 0.001405954360961914 nb_pixel_total : 69064 time to create 1 rle with old method : 0.07920980453491211 length of segment : 325 time for calcul the mask position with numpy : 0.0010399818420410156 nb_pixel_total : 40649 time to create 1 rle with old method : 0.050222158432006836 length of segment : 349 time for calcul the mask position with numpy : 0.0020694732666015625 nb_pixel_total : 51308 time to create 1 rle with old method : 0.05920290946960449 length of segment : 314 time for calcul the mask position with numpy : 0.0012252330780029297 nb_pixel_total : 33545 time to create 1 rle with old method : 0.054215192794799805 length of segment : 201 time for calcul the mask position with numpy : 0.002610445022583008 nb_pixel_total : 82403 time to create 1 rle with old method : 0.11296272277832031 length of segment : 358 time for calcul the mask position with numpy : 0.0023326873779296875 nb_pixel_total : 33551 time to create 1 rle with old method : 0.04045605659484863 length of segment : 227 time for calcul the mask position with numpy : 0.0003757476806640625 nb_pixel_total : 7734 time to create 1 rle with old method : 0.009810924530029297 length of segment : 92 time for calcul the mask position with numpy : 0.004226207733154297 nb_pixel_total : 115200 time to create 1 rle with old method : 0.13295459747314453 length of segment : 541 time for calcul the mask position with numpy : 0.002405405044555664 nb_pixel_total : 50073 time to create 1 rle with old method : 0.07132935523986816 length of segment : 274 time for calcul the mask position with numpy : 0.001130819320678711 nb_pixel_total : 26023 time to create 1 rle with old method : 0.0317227840423584 length of segment : 186 time for calcul the mask position with numpy : 0.0033490657806396484 nb_pixel_total : 87674 time to create 1 rle with old method : 0.1018211841583252 length of segment : 397 time for calcul the mask position with numpy : 0.005567789077758789 nb_pixel_total : 72775 time to create 1 rle with old method : 0.10599303245544434 length of segment : 373 time for calcul the mask position with numpy : 0.001302957534790039 nb_pixel_total : 35944 time to create 1 rle with old method : 0.04185962677001953 length of segment : 309 time for calcul the mask position with numpy : 0.0010821819305419922 nb_pixel_total : 28593 time to create 1 rle with old method : 0.03352475166320801 length of segment : 139 time for calcul the mask position with numpy : 0.0004322528839111328 nb_pixel_total : 8518 time to create 1 rle with old method : 0.010149478912353516 length of segment : 103 time for calcul the mask position with numpy : 0.0007052421569824219 nb_pixel_total : 20471 time to create 1 rle with old method : 0.025003433227539062 length of segment : 116 time for calcul the mask position with numpy : 0.0011515617370605469 nb_pixel_total : 10807 time to create 1 rle with old method : 0.013879776000976562 length of segment : 296 time for calcul the mask position with numpy : 0.0016262531280517578 nb_pixel_total : 18229 time to create 1 rle with old method : 0.020939111709594727 length of segment : 455 time for calcul the mask position with numpy : 0.0019352436065673828 nb_pixel_total : 64437 time to create 1 rle with old method : 0.07689976692199707 length of segment : 195 time for calcul the mask position with numpy : 0.000990152359008789 nb_pixel_total : 19356 time to create 1 rle with old method : 0.023081541061401367 length of segment : 156 time for calcul the mask position with numpy : 0.0006043910980224609 nb_pixel_total : 10863 time to create 1 rle with old method : 0.013089895248413086 length of segment : 101 time for calcul the mask position with numpy : 0.0007619857788085938 nb_pixel_total : 22015 time to create 1 rle with old method : 0.02568817138671875 length of segment : 176 time for calcul the mask position with numpy : 0.0014674663543701172 nb_pixel_total : 32407 time to create 1 rle with old method : 0.037120819091796875 length of segment : 288 time for calcul the mask position with numpy : 0.0009281635284423828 nb_pixel_total : 27194 time to create 1 rle with old method : 0.03173255920410156 length of segment : 187 time for calcul the mask position with numpy : 0.001050710678100586 nb_pixel_total : 19205 time to create 1 rle with old method : 0.02221059799194336 length of segment : 172 time for calcul the mask position with numpy : 0.0018875598907470703 nb_pixel_total : 53839 time to create 1 rle with old method : 0.06148695945739746 length of segment : 290 time for calcul the mask position with numpy : 0.0013566017150878906 nb_pixel_total : 20210 time to create 1 rle with old method : 0.023537635803222656 length of segment : 169 time for calcul the mask position with numpy : 0.0006654262542724609 nb_pixel_total : 17275 time to create 1 rle with old method : 0.02056741714477539 length of segment : 123 time for calcul the mask position with numpy : 0.0009610652923583984 nb_pixel_total : 15393 time to create 1 rle with old method : 0.017777204513549805 length of segment : 275 time for calcul the mask position with numpy : 0.0008542537689208984 nb_pixel_total : 16268 time to create 1 rle with old method : 0.018889188766479492 length of segment : 170 time for calcul the mask position with numpy : 0.0005931854248046875 nb_pixel_total : 20835 time to create 1 rle with old method : 0.023599863052368164 length of segment : 247 time for calcul the mask position with numpy : 0.0004906654357910156 nb_pixel_total : 10488 time to create 1 rle with old method : 0.012068986892700195 length of segment : 167 time for calcul the mask position with numpy : 0.001043081283569336 nb_pixel_total : 25297 time to create 1 rle with old method : 0.029812335968017578 length of segment : 283 time for calcul the mask position with numpy : 0.002888202667236328 nb_pixel_total : 52893 time to create 1 rle with old method : 0.06338953971862793 length of segment : 278 time for calcul the mask position with numpy : 0.0004215240478515625 nb_pixel_total : 12431 time to create 1 rle with old method : 0.014650583267211914 length of segment : 112 time for calcul the mask position with numpy : 0.0011668205261230469 nb_pixel_total : 42469 time to create 1 rle with old method : 0.04830050468444824 length of segment : 288 time for calcul the mask position with numpy : 0.0008635520935058594 nb_pixel_total : 20700 time to create 1 rle with old method : 0.023837566375732422 length of segment : 186 time for calcul the mask position with numpy : 0.0005314350128173828 nb_pixel_total : 18479 time to create 1 rle with old method : 0.02140212059020996 length of segment : 175 time for calcul the mask position with numpy : 0.0005502700805664062 nb_pixel_total : 15793 time to create 1 rle with old method : 0.018476247787475586 length of segment : 148 time for calcul the mask position with numpy : 0.0012645721435546875 nb_pixel_total : 56619 time to create 1 rle with old method : 0.06595849990844727 length of segment : 208 time for calcul the mask position with numpy : 0.0007922649383544922 nb_pixel_total : 30295 time to create 1 rle with old method : 0.034931182861328125 length of segment : 187 time for calcul the mask position with numpy : 0.0012204647064208984 nb_pixel_total : 35713 time to create 1 rle with old method : 0.04175543785095215 length of segment : 206 time for calcul the mask position with numpy : 0.0005993843078613281 nb_pixel_total : 17707 time to create 1 rle with old method : 0.02077770233154297 length of segment : 167 time for calcul the mask position with numpy : 0.004735231399536133 nb_pixel_total : 143439 time to create 1 rle with old method : 0.16214513778686523 length of segment : 851 time for calcul the mask position with numpy : 0.0019330978393554688 nb_pixel_total : 63751 time to create 1 rle with old method : 0.07230567932128906 length of segment : 312 time for calcul the mask position with numpy : 0.0002923011779785156 nb_pixel_total : 6151 time to create 1 rle with old method : 0.007270336151123047 length of segment : 94 time for calcul the mask position with numpy : 0.0009639263153076172 nb_pixel_total : 27453 time to create 1 rle with old method : 0.031520843505859375 length of segment : 227 time for calcul the mask position with numpy : 0.10669851303100586 nb_pixel_total : 445850 time to create 1 rle with new method : 0.04268670082092285 length of segment : 920 time for calcul the mask position with numpy : 0.009021997451782227 nb_pixel_total : 259340 time to create 1 rle with new method : 0.07796812057495117 length of segment : 829 time for calcul the mask position with numpy : 0.0004856586456298828 nb_pixel_total : 8593 time to create 1 rle with old method : 0.012325525283813477 length of segment : 87 time for calcul the mask position with numpy : 0.0004951953887939453 nb_pixel_total : 11194 time to create 1 rle with old method : 0.013329505920410156 length of segment : 143 time for calcul the mask position with numpy : 0.000972747802734375 nb_pixel_total : 22147 time to create 1 rle with old method : 0.026789426803588867 length of segment : 325 time for calcul the mask position with numpy : 0.0006062984466552734 nb_pixel_total : 19070 time to create 1 rle with old method : 0.022292375564575195 length of segment : 101 time for calcul the mask position with numpy : 0.0005381107330322266 nb_pixel_total : 18446 time to create 1 rle with old method : 0.02162909507751465 length of segment : 117 time for calcul the mask position with numpy : 0.0029599666595458984 nb_pixel_total : 115536 time to create 1 rle with old method : 0.13494563102722168 length of segment : 522 time for calcul the mask position with numpy : 0.0011684894561767578 nb_pixel_total : 33525 time to create 1 rle with old method : 0.038593292236328125 length of segment : 206 time for calcul the mask position with numpy : 0.0018875598907470703 nb_pixel_total : 15531 time to create 1 rle with old method : 0.019496440887451172 length of segment : 247 time for calcul the mask position with numpy : 0.0015120506286621094 nb_pixel_total : 52504 time to create 1 rle with old method : 0.05998110771179199 length of segment : 315 time for calcul the mask position with numpy : 0.0003330707550048828 nb_pixel_total : 13713 time to create 1 rle with old method : 0.016154050827026367 length of segment : 160 time for calcul the mask position with numpy : 0.00033736228942871094 nb_pixel_total : 9364 time to create 1 rle with old method : 0.0111541748046875 length of segment : 84 time for calcul the mask position with numpy : 0.0012183189392089844 nb_pixel_total : 42399 time to create 1 rle with old method : 0.04861044883728027 length of segment : 390 time for calcul the mask position with numpy : 0.0005192756652832031 nb_pixel_total : 14380 time to create 1 rle with old method : 0.016662120819091797 length of segment : 109 time for calcul the mask position with numpy : 0.00020933151245117188 nb_pixel_total : 11102 time to create 1 rle with old method : 0.013216972351074219 length of segment : 149 time for calcul the mask position with numpy : 0.0002963542938232422 nb_pixel_total : 13022 time to create 1 rle with old method : 0.015297651290893555 length of segment : 144 time for calcul the mask position with numpy : 0.0005524158477783203 nb_pixel_total : 20282 time to create 1 rle with old method : 0.023673295974731445 length of segment : 180 time for calcul the mask position with numpy : 0.0011816024780273438 nb_pixel_total : 35496 time to create 1 rle with old method : 0.04145383834838867 length of segment : 159 time for calcul the mask position with numpy : 0.0012264251708984375 nb_pixel_total : 34021 time to create 1 rle with old method : 0.039598941802978516 length of segment : 193 time for calcul the mask position with numpy : 0.0009114742279052734 nb_pixel_total : 24387 time to create 1 rle with old method : 0.02845025062561035 length of segment : 182 time for calcul the mask position with numpy : 0.0008835792541503906 nb_pixel_total : 25996 time to create 1 rle with old method : 0.030459880828857422 length of segment : 127 time for calcul the mask position with numpy : 0.0007507801055908203 nb_pixel_total : 19077 time to create 1 rle with old method : 0.023649215698242188 length of segment : 222 time for calcul the mask position with numpy : 0.0018546581268310547 nb_pixel_total : 38350 time to create 1 rle with old method : 0.05528593063354492 length of segment : 417 time for calcul the mask position with numpy : 0.0012369155883789062 nb_pixel_total : 35904 time to create 1 rle with old method : 0.04146718978881836 length of segment : 176 time for calcul the mask position with numpy : 0.002088308334350586 nb_pixel_total : 33754 time to create 1 rle with old method : 0.038559675216674805 length of segment : 492 time for calcul the mask position with numpy : 0.001264333724975586 nb_pixel_total : 24761 time to create 1 rle with old method : 0.02872633934020996 length of segment : 192 time for calcul the mask position with numpy : 0.0005660057067871094 nb_pixel_total : 10512 time to create 1 rle with old method : 0.012395620346069336 length of segment : 149 time for calcul the mask position with numpy : 0.0007462501525878906 nb_pixel_total : 12725 time to create 1 rle with old method : 0.014869928359985352 length of segment : 211 time for calcul the mask position with numpy : 0.0007691383361816406 nb_pixel_total : 19483 time to create 1 rle with old method : 0.022298574447631836 length of segment : 217 time for calcul the mask position with numpy : 0.0004639625549316406 nb_pixel_total : 6506 time to create 1 rle with old method : 0.008028030395507812 length of segment : 79 time for calcul the mask position with numpy : 0.0002665519714355469 nb_pixel_total : 8848 time to create 1 rle with old method : 0.010627031326293945 length of segment : 123 time for calcul the mask position with numpy : 0.00074005126953125 nb_pixel_total : 11845 time to create 1 rle with old method : 0.014050483703613281 length of segment : 135 time for calcul the mask position with numpy : 0.0003402233123779297 nb_pixel_total : 7716 time to create 1 rle with old method : 0.009263277053833008 length of segment : 81 time for calcul the mask position with numpy : 0.0003342628479003906 nb_pixel_total : 4119 time to create 1 rle with old method : 0.004948854446411133 length of segment : 90 time for calcul the mask position with numpy : 0.004808902740478516 nb_pixel_total : 123657 time to create 1 rle with old method : 0.1437838077545166 length of segment : 575 time for calcul the mask position with numpy : 0.0007178783416748047 nb_pixel_total : 18945 time to create 1 rle with old method : 0.0245208740234375 length of segment : 139 time for calcul the mask position with numpy : 0.002429962158203125 nb_pixel_total : 59265 time to create 1 rle with old method : 0.06726741790771484 length of segment : 416 time for calcul the mask position with numpy : 0.011382341384887695 nb_pixel_total : 312370 time to create 1 rle with new method : 0.04447531700134277 length of segment : 786 time for calcul the mask position with numpy : 0.0011229515075683594 nb_pixel_total : 28017 time to create 1 rle with old method : 0.03453969955444336 length of segment : 169 time for calcul the mask position with numpy : 0.005415439605712891 nb_pixel_total : 182255 time to create 1 rle with new method : 0.00987863540649414 length of segment : 537 time for calcul the mask position with numpy : 0.0022192001342773438 nb_pixel_total : 68151 time to create 1 rle with old method : 0.0793464183807373 length of segment : 287 time for calcul the mask position with numpy : 0.0004470348358154297 nb_pixel_total : 7661 time to create 1 rle with old method : 0.009256839752197266 length of segment : 122 time for calcul the mask position with numpy : 0.000400543212890625 nb_pixel_total : 12658 time to create 1 rle with old method : 0.015430450439453125 length of segment : 82 time for calcul the mask position with numpy : 0.0015985965728759766 nb_pixel_total : 59614 time to create 1 rle with old method : 0.06844639778137207 length of segment : 366 time for calcul the mask position with numpy : 0.0016601085662841797 nb_pixel_total : 55366 time to create 1 rle with old method : 0.06332230567932129 length of segment : 381 time for calcul the mask position with numpy : 0.0002548694610595703 nb_pixel_total : 7623 time to create 1 rle with old method : 0.009193658828735352 length of segment : 68 time for calcul the mask position with numpy : 0.011505126953125 nb_pixel_total : 472740 time to create 1 rle with new method : 0.06845688819885254 length of segment : 795 time for calcul the mask position with numpy : 0.0006749629974365234 nb_pixel_total : 17182 time to create 1 rle with old method : 0.020078420639038086 length of segment : 163 time for calcul the mask position with numpy : 0.008265018463134766 nb_pixel_total : 141971 time to create 1 rle with old method : 0.164381742477417 length of segment : 744 time for calcul the mask position with numpy : 0.005564689636230469 nb_pixel_total : 187893 time to create 1 rle with new method : 0.009107112884521484 length of segment : 563 time for calcul the mask position with numpy : 0.001684427261352539 nb_pixel_total : 30357 time to create 1 rle with old method : 0.034685611724853516 length of segment : 337 time for calcul the mask position with numpy : 0.0008857250213623047 nb_pixel_total : 24580 time to create 1 rle with old method : 0.02865123748779297 length of segment : 189 time for calcul the mask position with numpy : 0.0020716190338134766 nb_pixel_total : 64910 time to create 1 rle with old method : 0.0748283863067627 length of segment : 401 time for calcul the mask position with numpy : 0.001432180404663086 nb_pixel_total : 38583 time to create 1 rle with old method : 0.044763803482055664 length of segment : 234 time for calcul the mask position with numpy : 0.003098726272583008 nb_pixel_total : 86833 time to create 1 rle with old method : 0.09988141059875488 length of segment : 358 time for calcul the mask position with numpy : 0.0034897327423095703 nb_pixel_total : 104937 time to create 1 rle with old method : 0.12005090713500977 length of segment : 418 time for calcul the mask position with numpy : 0.003929853439331055 nb_pixel_total : 103818 time to create 1 rle with old method : 0.11924123764038086 length of segment : 387 time for calcul the mask position with numpy : 0.0025014877319335938 nb_pixel_total : 64609 time to create 1 rle with old method : 0.07412433624267578 length of segment : 498 time for calcul the mask position with numpy : 0.0051844120025634766 nb_pixel_total : 206487 time to create 1 rle with new method : 0.008414983749389648 length of segment : 527 time for calcul the mask position with numpy : 0.0020902156829833984 nb_pixel_total : 53781 time to create 1 rle with old method : 0.06810498237609863 length of segment : 460 time for calcul the mask position with numpy : 0.003286600112915039 nb_pixel_total : 64055 time to create 1 rle with old method : 0.0821840763092041 length of segment : 323 time for calcul the mask position with numpy : 0.0003857612609863281 nb_pixel_total : 11573 time to create 1 rle with old method : 0.013608694076538086 length of segment : 156 time for calcul the mask position with numpy : 0.0009288787841796875 nb_pixel_total : 28686 time to create 1 rle with old method : 0.033577680587768555 length of segment : 141 time for calcul the mask position with numpy : 0.0007617473602294922 nb_pixel_total : 25602 time to create 1 rle with old method : 0.029969215393066406 length of segment : 196 time for calcul the mask position with numpy : 0.00057220458984375 nb_pixel_total : 19277 time to create 1 rle with old method : 0.022968292236328125 length of segment : 148 time for calcul the mask position with numpy : 0.0018901824951171875 nb_pixel_total : 59210 time to create 1 rle with old method : 0.06745719909667969 length of segment : 315 time for calcul the mask position with numpy : 0.0015633106231689453 nb_pixel_total : 52188 time to create 1 rle with old method : 0.05827474594116211 length of segment : 408 time for calcul the mask position with numpy : 0.00020170211791992188 nb_pixel_total : 9036 time to create 1 rle with old method : 0.010892152786254883 length of segment : 93 time for calcul the mask position with numpy : 0.003331899642944336 nb_pixel_total : 103181 time to create 1 rle with old method : 0.11661863327026367 length of segment : 441 time spent for convertir_results : 12.475374937057495 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 136 chid ids of type : 3594 Number RLEs to save : 37493 save missing photos in datou_result : time spend for datou_step_exec : 67.1551125049591 time spend to save output : 2.2934887409210205 total time spend for step 1 : 69.44860124588013 step2:crop_condition Tue May 13 22:31:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 136 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 90 About to insert : list_path_to_insert length 90 new photo from crops ! About to upload 90 photos upload in portfolio : 3736932 init cache_photo without model_param we have 90 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747168324_363352 we have uploaded 90 photos in the portfolio 3736932 time of upload the photos Elapsed time : 36.17112898826599 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747168365_363352 we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 9.383796215057373 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747168376_363352 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5439369678497314 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 27 About to insert : list_path_to_insert length 27 new photo from crops ! About to upload 27 photos upload in portfolio : 3736932 init cache_photo without model_param we have 27 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747168386_363352 we have uploaded 27 photos in the portfolio 3736932 time of upload the photos Elapsed time : 6.439178228378296 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747168394_363352 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7198348045349121 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747168396_363352 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7233762741088867 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1357866036, 1357866030, 1357866024, 1357865892, 1357865873, 1357865805, 1357865723, 1357865473, 1357863818] Looping around the photos to save general results len do output : 136 /1357974933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357974999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357975459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866036', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866030', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866024', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865892', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865873', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865805', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865723', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865473', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357863818', None, None, None, None, None, '2843491') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 417 time used for this insertion : 0.029664039611816406 save_final save missing photos in datou_result : time spend for datou_step_exec : 96.38799452781677 time spend to save output : 0.035086631774902344 total time spend for step 2 : 96.42308115959167 step3:rle_unique_nms_with_priority Tue May 13 22:33:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 136 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 7.257429838180542 time for calcul the mask position with numpy : 0.42264485359191895 nb_pixel_total : 5861052 time to create 1 rle with new method : 1.3006365299224854 time for calcul the mask position with numpy : 0.024504899978637695 nb_pixel_total : 10594 time to create 1 rle with old method : 0.019612550735473633 time for calcul the mask position with numpy : 0.03234529495239258 nb_pixel_total : 19321 time to create 1 rle with old method : 0.02653026580810547 time for calcul the mask position with numpy : 0.024865150451660156 nb_pixel_total : 76415 time to create 1 rle with old method : 0.09467196464538574 time for calcul the mask position with numpy : 0.02212977409362793 nb_pixel_total : 18246 time to create 1 rle with old method : 0.021248340606689453 time for calcul the mask position with numpy : 0.023978233337402344 nb_pixel_total : 40081 time to create 1 rle with old method : 0.04757070541381836 time for calcul the mask position with numpy : 0.02801227569580078 nb_pixel_total : 90600 time to create 1 rle with old method : 0.14475512504577637 time for calcul the mask position with numpy : 0.02262592315673828 nb_pixel_total : 69827 time to create 1 rle with old method : 0.09144282341003418 time for calcul the mask position with numpy : 0.02274465560913086 nb_pixel_total : 80085 time to create 1 rle with old method : 0.08934664726257324 time for calcul the mask position with numpy : 0.038809776306152344 nb_pixel_total : 61923 time to create 1 rle with old method : 0.07500505447387695 time for calcul the mask position with numpy : 0.037497758865356445 nb_pixel_total : 129558 time to create 1 rle with old method : 0.21378684043884277 time for calcul the mask position with numpy : 0.03577446937561035 nb_pixel_total : 63038 time to create 1 rle with old method : 0.08780050277709961 time for calcul the mask position with numpy : 0.0359196662902832 nb_pixel_total : 80160 time to create 1 rle with old method : 0.08942627906799316 time for calcul the mask position with numpy : 0.03455233573913574 nb_pixel_total : 20575 time to create 1 rle with old method : 0.02357959747314453 time for calcul the mask position with numpy : 0.03833651542663574 nb_pixel_total : 428765 time to create 1 rle with new method : 0.8404161930084229 create new chi : 4.072370290756226 time to delete rle : 0.0199434757232666 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 11210 TO DO : save crop sub photo not yet done ! save time : 0.7379143238067627 nb_obj : 13 nb_hashtags : 3 time to prepare the origin masks : 5.48249077796936 time for calcul the mask position with numpy : 0.5041215419769287 nb_pixel_total : 6479390 time to create 1 rle with new method : 1.0393152236938477 time for calcul the mask position with numpy : 0.03927874565124512 nb_pixel_total : 115200 time to create 1 rle with old method : 0.1290428638458252 time for calcul the mask position with numpy : 0.0352323055267334 nb_pixel_total : 7734 time to create 1 rle with old method : 0.008810043334960938 time for calcul the mask position with numpy : 0.042722463607788086 nb_pixel_total : 33551 time to create 1 rle with old method : 0.04532480239868164 time for calcul the mask position with numpy : 0.027618408203125 nb_pixel_total : 82403 time to create 1 rle with old method : 0.09535431861877441 time for calcul the mask position with numpy : 0.02131819725036621 nb_pixel_total : 33545 time to create 1 rle with old method : 0.03753924369812012 time for calcul the mask position with numpy : 0.02164459228515625 nb_pixel_total : 51308 time to create 1 rle with old method : 0.0597531795501709 time for calcul the mask position with numpy : 0.02285599708557129 nb_pixel_total : 40649 time to create 1 rle with old method : 0.06115841865539551 time for calcul the mask position with numpy : 0.024121761322021484 nb_pixel_total : 69064 time to create 1 rle with old method : 0.07962989807128906 time for calcul the mask position with numpy : 0.021976232528686523 nb_pixel_total : 11449 time to create 1 rle with old method : 0.012969493865966797 time for calcul the mask position with numpy : 0.024701356887817383 nb_pixel_total : 29386 time to create 1 rle with old method : 0.03352928161621094 time for calcul the mask position with numpy : 0.023406267166137695 nb_pixel_total : 44382 time to create 1 rle with old method : 0.049920082092285156 time for calcul the mask position with numpy : 0.02302074432373047 nb_pixel_total : 20142 time to create 1 rle with old method : 0.022656679153442383 time for calcul the mask position with numpy : 0.023487329483032227 nb_pixel_total : 32037 time to create 1 rle with old method : 0.03603649139404297 create new chi : 2.616237163543701 time to delete rle : 0.0014524459838867188 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 8932 TO DO : save crop sub photo not yet done ! save time : 0.6448192596435547 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 3.322915554046631 time for calcul the mask position with numpy : 0.42612791061401367 nb_pixel_total : 6709362 time to create 1 rle with new method : 0.6206099987030029 time for calcul the mask position with numpy : 0.03367114067077637 nb_pixel_total : 10807 time to create 1 rle with old method : 0.011814594268798828 time for calcul the mask position with numpy : 0.0314488410949707 nb_pixel_total : 20471 time to create 1 rle with old method : 0.021931886672973633 time for calcul the mask position with numpy : 0.028061389923095703 nb_pixel_total : 8518 time to create 1 rle with old method : 0.009828805923461914 time for calcul the mask position with numpy : 0.027399778366088867 nb_pixel_total : 28593 time to create 1 rle with old method : 0.032251834869384766 time for calcul the mask position with numpy : 0.02057480812072754 nb_pixel_total : 35944 time to create 1 rle with old method : 0.04031777381896973 time for calcul the mask position with numpy : 0.021135568618774414 nb_pixel_total : 72775 time to create 1 rle with old method : 0.07724165916442871 time for calcul the mask position with numpy : 0.024285554885864258 nb_pixel_total : 87674 time to create 1 rle with old method : 0.09346675872802734 time for calcul the mask position with numpy : 0.032082319259643555 nb_pixel_total : 26023 time to create 1 rle with old method : 0.026925086975097656 time for calcul the mask position with numpy : 0.03180646896362305 nb_pixel_total : 50073 time to create 1 rle with old method : 0.05161023139953613 create new chi : 1.6990244388580322 time to delete rle : 0.0010349750518798828 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 6546 TO DO : save crop sub photo not yet done ! save time : 0.5677597522735596 nb_obj : 18 nb_hashtags : 4 time to prepare the origin masks : 6.434468984603882 time for calcul the mask position with numpy : 0.7313294410705566 nb_pixel_total : 6596153 time to create 1 rle with new method : 0.7846531867980957 time for calcul the mask position with numpy : 0.02025437355041504 nb_pixel_total : 12431 time to create 1 rle with old method : 0.013514995574951172 time for calcul the mask position with numpy : 0.020925045013427734 nb_pixel_total : 52893 time to create 1 rle with old method : 0.05762171745300293 time for calcul the mask position with numpy : 0.03373527526855469 nb_pixel_total : 25297 time to create 1 rle with old method : 0.02835988998413086 time for calcul the mask position with numpy : 0.03413891792297363 nb_pixel_total : 10488 time to create 1 rle with old method : 0.011449337005615234 time for calcul the mask position with numpy : 0.032544851303100586 nb_pixel_total : 16287 time to create 1 rle with old method : 0.018100738525390625 time for calcul the mask position with numpy : 0.03215456008911133 nb_pixel_total : 16268 time to create 1 rle with old method : 0.01794290542602539 time for calcul the mask position with numpy : 0.03239703178405762 nb_pixel_total : 15393 time to create 1 rle with old method : 0.016857385635375977 time for calcul the mask position with numpy : 0.032469749450683594 nb_pixel_total : 17275 time to create 1 rle with old method : 0.019379138946533203 time for calcul the mask position with numpy : 0.03241252899169922 nb_pixel_total : 20210 time to create 1 rle with old method : 0.021811962127685547 time for calcul the mask position with numpy : 0.0217742919921875 nb_pixel_total : 53839 time to create 1 rle with old method : 0.06222033500671387 time for calcul the mask position with numpy : 0.021145343780517578 nb_pixel_total : 19205 time to create 1 rle with old method : 0.020859479904174805 time for calcul the mask position with numpy : 0.02567577362060547 nb_pixel_total : 27194 time to create 1 rle with old method : 0.029272794723510742 time for calcul the mask position with numpy : 0.021364212036132812 nb_pixel_total : 32407 time to create 1 rle with old method : 0.0362086296081543 time for calcul the mask position with numpy : 0.02206730842590332 nb_pixel_total : 22015 time to create 1 rle with old method : 0.02508687973022461 time for calcul the mask position with numpy : 0.02400374412536621 nb_pixel_total : 10863 time to create 1 rle with old method : 0.01291656494140625 time for calcul the mask position with numpy : 0.02560591697692871 nb_pixel_total : 19356 time to create 1 rle with old method : 0.02214789390563965 time for calcul the mask position with numpy : 0.02449941635131836 nb_pixel_total : 64437 time to create 1 rle with old method : 0.07257747650146484 time for calcul the mask position with numpy : 0.021669626235961914 nb_pixel_total : 18229 time to create 1 rle with old method : 0.020827770233154297 create new chi : 2.5402421951293945 time to delete rle : 0.0022466182708740234 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 9710 TO DO : save crop sub photo not yet done ! save time : 0.7255170345306396 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 5.4399285316467285 time for calcul the mask position with numpy : 0.6370751857757568 nb_pixel_total : 6633318 time to create 1 rle with new method : 0.803617000579834 time for calcul the mask position with numpy : 0.02528977394104004 nb_pixel_total : 6151 time to create 1 rle with old method : 0.007013559341430664 time for calcul the mask position with numpy : 0.025672197341918945 nb_pixel_total : 63751 time to create 1 rle with old method : 0.07479429244995117 time for calcul the mask position with numpy : 0.02427959442138672 nb_pixel_total : 109245 time to create 1 rle with old method : 0.12161803245544434 time for calcul the mask position with numpy : 0.022570371627807617 nb_pixel_total : 17707 time to create 1 rle with old method : 0.019359827041625977 time for calcul the mask position with numpy : 0.022626638412475586 nb_pixel_total : 35713 time to create 1 rle with old method : 0.03980422019958496 time for calcul the mask position with numpy : 0.021561384201049805 nb_pixel_total : 30295 time to create 1 rle with old method : 0.034186363220214844 time for calcul the mask position with numpy : 0.021810054779052734 nb_pixel_total : 56619 time to create 1 rle with old method : 0.06346535682678223 time for calcul the mask position with numpy : 0.03427243232727051 nb_pixel_total : 15793 time to create 1 rle with old method : 0.016984939575195312 time for calcul the mask position with numpy : 0.019889354705810547 nb_pixel_total : 18479 time to create 1 rle with old method : 0.02019357681274414 time for calcul the mask position with numpy : 0.020910263061523438 nb_pixel_total : 20700 time to create 1 rle with old method : 0.021888017654418945 time for calcul the mask position with numpy : 0.02077317237854004 nb_pixel_total : 42469 time to create 1 rle with old method : 0.04677891731262207 create new chi : 2.2223470211029053 time to delete rle : 0.0012068748474121094 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++Number RLEs to save : 7560 TO DO : save crop sub photo not yet done ! save time : 0.5373847484588623 nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 7.6619110107421875 time for calcul the mask position with numpy : 0.4530467987060547 nb_pixel_total : 5863247 time to create 1 rle with new method : 0.8169071674346924 time for calcul the mask position with numpy : 0.023792028427124023 nb_pixel_total : 35496 time to create 1 rle with old method : 0.04205775260925293 time for calcul the mask position with numpy : 0.023257732391357422 nb_pixel_total : 20282 time to create 1 rle with old method : 0.022130966186523438 time for calcul the mask position with numpy : 0.021139860153198242 nb_pixel_total : 13022 time to create 1 rle with old method : 0.014345407485961914 time for calcul the mask position with numpy : 0.02109503746032715 nb_pixel_total : 9758 time to create 1 rle with old method : 0.010892391204833984 time for calcul the mask position with numpy : 0.02083730697631836 nb_pixel_total : 14380 time to create 1 rle with old method : 0.015870094299316406 time for calcul the mask position with numpy : 0.020478010177612305 nb_pixel_total : 42399 time to create 1 rle with old method : 0.0451962947845459 time for calcul the mask position with numpy : 0.021236181259155273 nb_pixel_total : 9364 time to create 1 rle with old method : 0.010095834732055664 time for calcul the mask position with numpy : 0.0218656063079834 nb_pixel_total : 13103 time to create 1 rle with old method : 0.014145374298095703 time for calcul the mask position with numpy : 0.020635604858398438 nb_pixel_total : 52504 time to create 1 rle with old method : 0.0572969913482666 time for calcul the mask position with numpy : 0.021991729736328125 nb_pixel_total : 15531 time to create 1 rle with old method : 0.017714738845825195 time for calcul the mask position with numpy : 0.021912813186645508 nb_pixel_total : 33525 time to create 1 rle with old method : 0.0365598201751709 time for calcul the mask position with numpy : 0.021953344345092773 nb_pixel_total : 115536 time to create 1 rle with old method : 0.12465405464172363 time for calcul the mask position with numpy : 0.021662235260009766 nb_pixel_total : 18446 time to create 1 rle with old method : 0.020535707473754883 time for calcul the mask position with numpy : 0.02184772491455078 nb_pixel_total : 19070 time to create 1 rle with old method : 0.020864248275756836 time for calcul the mask position with numpy : 0.020845651626586914 nb_pixel_total : 22147 time to create 1 rle with old method : 0.02419424057006836 time for calcul the mask position with numpy : 0.02080678939819336 nb_pixel_total : 11194 time to create 1 rle with old method : 0.012189865112304688 time for calcul the mask position with numpy : 0.02138066291809082 nb_pixel_total : 8593 time to create 1 rle with old method : 0.009544849395751953 time for calcul the mask position with numpy : 0.023245811462402344 nb_pixel_total : 259340 time to create 1 rle with new method : 0.8647019863128662 time for calcul the mask position with numpy : 0.024547100067138672 nb_pixel_total : 445850 time to create 1 rle with new method : 0.7208828926086426 time for calcul the mask position with numpy : 0.022000789642333984 nb_pixel_total : 27453 time to create 1 rle with old method : 0.030007362365722656 create new chi : 3.906219482421875 time to delete rle : 0.0020241737365722656 batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++Number RLEs to save : 12884 TO DO : save crop sub photo not yet done ! save time : 0.7912452220916748 nb_obj : 18 nb_hashtags : 3 time to prepare the origin masks : 7.034019231796265 time for calcul the mask position with numpy : 0.6228554248809814 nb_pixel_total : 6598216 time to create 1 rle with new method : 0.7559642791748047 time for calcul the mask position with numpy : 0.034668684005737305 nb_pixel_total : 18945 time to create 1 rle with old method : 0.0214078426361084 time for calcul the mask position with numpy : 0.023133516311645508 nb_pixel_total : 122340 time to create 1 rle with old method : 0.133897066116333 time for calcul the mask position with numpy : 0.02062225341796875 nb_pixel_total : 4119 time to create 1 rle with old method : 0.004578828811645508 time for calcul the mask position with numpy : 0.02100658416748047 nb_pixel_total : 7716 time to create 1 rle with old method : 0.008823394775390625 time for calcul the mask position with numpy : 0.020308732986450195 nb_pixel_total : 11845 time to create 1 rle with old method : 0.013111352920532227 time for calcul the mask position with numpy : 0.02145862579345703 nb_pixel_total : 1583 time to create 1 rle with old method : 0.0019075870513916016 time for calcul the mask position with numpy : 0.022259950637817383 nb_pixel_total : 6506 time to create 1 rle with old method : 0.007376670837402344 time for calcul the mask position with numpy : 0.0220794677734375 nb_pixel_total : 19483 time to create 1 rle with old method : 0.022009611129760742 time for calcul the mask position with numpy : 0.02185821533203125 nb_pixel_total : 12725 time to create 1 rle with old method : 0.014100074768066406 time for calcul the mask position with numpy : 0.02216339111328125 nb_pixel_total : 10512 time to create 1 rle with old method : 0.013452529907226562 time for calcul the mask position with numpy : 0.024374008178710938 nb_pixel_total : 24761 time to create 1 rle with old method : 0.02879190444946289 time for calcul the mask position with numpy : 0.021538257598876953 nb_pixel_total : 33754 time to create 1 rle with old method : 0.03634071350097656 time for calcul the mask position with numpy : 0.022969484329223633 nb_pixel_total : 35904 time to create 1 rle with old method : 0.04413247108459473 time for calcul the mask position with numpy : 0.023148775100708008 nb_pixel_total : 38350 time to create 1 rle with old method : 0.041925907135009766 time for calcul the mask position with numpy : 0.020781755447387695 nb_pixel_total : 19077 time to create 1 rle with old method : 0.021157503128051758 time for calcul the mask position with numpy : 0.021518468856811523 nb_pixel_total : 25996 time to create 1 rle with old method : 0.028717041015625 time for calcul the mask position with numpy : 0.02116084098815918 nb_pixel_total : 24387 time to create 1 rle with old method : 0.026443004608154297 time for calcul the mask position with numpy : 0.022513628005981445 nb_pixel_total : 34021 time to create 1 rle with old method : 0.037035465240478516 create new chi : 2.3306941986083984 time to delete rle : 0.0014853477478027344 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 9553 TO DO : save crop sub photo not yet done ! save time : 0.6360547542572021 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 6.154988527297974 time for calcul the mask position with numpy : 0.4836771488189697 nb_pixel_total : 5437474 time to create 1 rle with new method : 0.6555027961730957 time for calcul the mask position with numpy : 0.021569490432739258 nb_pixel_total : 187893 time to create 1 rle with new method : 0.561260461807251 time for calcul the mask position with numpy : 0.022507905960083008 nb_pixel_total : 141971 time to create 1 rle with old method : 0.1813802719116211 time for calcul the mask position with numpy : 0.020989418029785156 nb_pixel_total : 17182 time to create 1 rle with old method : 0.018938541412353516 time for calcul the mask position with numpy : 0.024236440658569336 nb_pixel_total : 472740 time to create 1 rle with new method : 0.6482150554656982 time for calcul the mask position with numpy : 0.0231473445892334 nb_pixel_total : 7623 time to create 1 rle with old method : 0.009936809539794922 time for calcul the mask position with numpy : 0.034623146057128906 nb_pixel_total : 55366 time to create 1 rle with old method : 0.06123471260070801 time for calcul the mask position with numpy : 0.036328792572021484 nb_pixel_total : 59614 time to create 1 rle with old method : 0.06918907165527344 time for calcul the mask position with numpy : 0.034861087799072266 nb_pixel_total : 12658 time to create 1 rle with old method : 0.01432180404663086 time for calcul the mask position with numpy : 0.0354762077331543 nb_pixel_total : 7661 time to create 1 rle with old method : 0.008676767349243164 time for calcul the mask position with numpy : 0.03549957275390625 nb_pixel_total : 68151 time to create 1 rle with old method : 0.07497930526733398 time for calcul the mask position with numpy : 0.03421664237976074 nb_pixel_total : 182255 time to create 1 rle with new method : 0.6313326358795166 time for calcul the mask position with numpy : 0.032836198806762695 nb_pixel_total : 28017 time to create 1 rle with old method : 0.03202486038208008 time for calcul the mask position with numpy : 0.03564620018005371 nb_pixel_total : 312370 time to create 1 rle with new method : 0.6042375564575195 time for calcul the mask position with numpy : 0.025542020797729492 nb_pixel_total : 59265 time to create 1 rle with old method : 0.06424546241760254 create new chi : 4.665982723236084 time to delete rle : 0.002008676528930664 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 13118 TO DO : save crop sub photo not yet done ! save time : 0.8774049282073975 nb_obj : 19 nb_hashtags : 4 time to prepare the origin masks : 7.7872843742370605 time for calcul the mask position with numpy : 0.5187778472900391 nb_pixel_total : 5914626 time to create 1 rle with new method : 0.8641481399536133 time for calcul the mask position with numpy : 0.022737979888916016 nb_pixel_total : 103181 time to create 1 rle with old method : 0.11694693565368652 time for calcul the mask position with numpy : 0.022685766220092773 nb_pixel_total : 3121 time to create 1 rle with old method : 0.0037326812744140625 time for calcul the mask position with numpy : 0.023606300354003906 nb_pixel_total : 52188 time to create 1 rle with old method : 0.05791783332824707 time for calcul the mask position with numpy : 0.022115230560302734 nb_pixel_total : 59198 time to create 1 rle with old method : 0.09213685989379883 time for calcul the mask position with numpy : 0.02177906036376953 nb_pixel_total : 19277 time to create 1 rle with old method : 0.021710634231567383 time for calcul the mask position with numpy : 0.02222132682800293 nb_pixel_total : 25602 time to create 1 rle with old method : 0.031734466552734375 time for calcul the mask position with numpy : 0.028562545776367188 nb_pixel_total : 28686 time to create 1 rle with old method : 0.03825187683105469 time for calcul the mask position with numpy : 0.02347254753112793 nb_pixel_total : 11573 time to create 1 rle with old method : 0.013106584548950195 time for calcul the mask position with numpy : 0.026795387268066406 nb_pixel_total : 64055 time to create 1 rle with old method : 0.07418417930603027 time for calcul the mask position with numpy : 0.021374225616455078 nb_pixel_total : 53519 time to create 1 rle with old method : 0.059906005859375 time for calcul the mask position with numpy : 0.02313232421875 nb_pixel_total : 196587 time to create 1 rle with new method : 0.5615003108978271 time for calcul the mask position with numpy : 0.021291017532348633 nb_pixel_total : 64609 time to create 1 rle with old method : 0.0686178207397461 time for calcul the mask position with numpy : 0.034246206283569336 nb_pixel_total : 103818 time to create 1 rle with old method : 0.11152529716491699 time for calcul the mask position with numpy : 0.033150672912597656 nb_pixel_total : 104937 time to create 1 rle with old method : 0.11308693885803223 time for calcul the mask position with numpy : 0.03268885612487793 nb_pixel_total : 86833 time to create 1 rle with old method : 0.09234905242919922 time for calcul the mask position with numpy : 0.0316014289855957 nb_pixel_total : 38583 time to create 1 rle with old method : 0.04083609580993652 time for calcul the mask position with numpy : 0.03640937805175781 nb_pixel_total : 64910 time to create 1 rle with old method : 0.0702362060546875 time for calcul the mask position with numpy : 0.031847238540649414 nb_pixel_total : 24580 time to create 1 rle with old method : 0.026775598526000977 time for calcul the mask position with numpy : 0.022317171096801758 nb_pixel_total : 30357 time to create 1 rle with old method : 0.033437490463256836 create new chi : 3.576798439025879 time to delete rle : 0.00218963623046875 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 13897 TO DO : save crop sub photo not yet done ! save time : 0.9329736232757568 map_output_result : {1357866036: (0.0, 'Should be the crop_list due to order', 0), 1357866030: (0.0, 'Should be the crop_list due to order', 0), 1357866024: (0.0, 'Should be the crop_list due to order', 0), 1357865892: (0.0, 'Should be the crop_list due to order', 0), 1357865873: (0.0, 'Should be the crop_list due to order', 0), 1357865805: (0.0, 'Should be the crop_list due to order', 0), 1357865723: (0.0, 'Should be the crop_list due to order', 0), 1357865473: (0.0, 'Should be the crop_list due to order', 0), 1357863818: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1357866036, 1357866030, 1357866024, 1357865892, 1357865873, 1357865805, 1357865723, 1357865473, 1357863818] Looping around the photos to save general results len do output : 9 /1357866036.Didn't retrieve data . /1357866030.Didn't retrieve data . /1357866024.Didn't retrieve data . /1357865892.Didn't retrieve data . /1357865873.Didn't retrieve data . /1357865805.Didn't retrieve data . /1357865723.Didn't retrieve data . /1357865473.Didn't retrieve data . /1357863818.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866036', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866030', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866024', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865892', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865873', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865805', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865723', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865473', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357863818', None, None, None, None, None, '2843491') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.014070987701416016 save_final save missing photos in datou_result : time spend for datou_step_exec : 91.9706461429596 time spend to save output : 0.014682769775390625 total time spend for step 3 : 91.98532891273499 step4:ventilate_hashtags_in_portfolio Tue May 13 22:34:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22934121 get user id for portfolio 22934121 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934121 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','pet_clair','autre','flou','papier','environnement','pet_fonce','background','pehd','metal','mal_croppe')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934121 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','pet_clair','autre','flou','papier','environnement','pet_fonce','background','pehd','metal','mal_croppe')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934121 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('carton','pet_clair','autre','flou','papier','environnement','pet_fonce','background','pehd','metal','mal_croppe')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/22937974,22937975,22937976,22937977,22937978,22937979,22937980,22937981,22937982,22937983,22937984?tags=carton,pet_clair,autre,flou,papier,environnement,pet_fonce,background,pehd,metal,mal_croppe Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1357866036, 1357866030, 1357866024, 1357865892, 1357865873, 1357865805, 1357865723, 1357865473, 1357863818] Looping around the photos to save general results len do output : 1 /22934121. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866036', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866030', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866024', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865892', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865873', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865805', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865723', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865473', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357863818', None, None, None, None, None, '2843491') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.017310142517089844 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5457067489624023 time spend to save output : 0.01757669448852539 total time spend for step 4 : 1.5632834434509277 step5:final Tue May 13 22:34:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1357866036: ('0.11598221400185589',), 1357866030: ('0.11598221400185589',), 1357866024: ('0.11598221400185589',), 1357865892: ('0.11598221400185589',), 1357865873: ('0.11598221400185589',), 1357865805: ('0.11598221400185589',), 1357865723: ('0.11598221400185589',), 1357865473: ('0.11598221400185589',), 1357863818: ('0.11598221400185589',)} new output for save of step final : {1357866036: ('0.11598221400185589',), 1357866030: ('0.11598221400185589',), 1357866024: ('0.11598221400185589',), 1357865892: ('0.11598221400185589',), 1357865873: ('0.11598221400185589',), 1357865805: ('0.11598221400185589',), 1357865723: ('0.11598221400185589',), 1357865473: ('0.11598221400185589',), 1357863818: ('0.11598221400185589',)} [1357866036, 1357866030, 1357866024, 1357865892, 1357865873, 1357865805, 1357865723, 1357865473, 1357863818] Looping around the photos to save general results len do output : 9 /1357866036.Didn't retrieve data . /1357866030.Didn't retrieve data . /1357866024.Didn't retrieve data . /1357865892.Didn't retrieve data . /1357865873.Didn't retrieve data . /1357865805.Didn't retrieve data . /1357865723.Didn't retrieve data . /1357865473.Didn't retrieve data . /1357863818.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866036', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866030', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866024', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865892', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865873', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865805', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865723', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865473', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357863818', None, None, None, None, None, '2843491') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.013559818267822266 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10642576217651367 time spend to save output : 0.014039754867553711 total time spend for step 5 : 0.12046551704406738 step6:blur_detection Tue May 13 22:34:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76.jpg resize: (2160, 3264) 1357866036 -1.488323768867265 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c.jpg resize: (2160, 3264) 1357866030 -0.8332922896356084 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc.jpg resize: (2160, 3264) 1357866024 -1.4653933444232876 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2.jpg resize: (2160, 3264) 1357865892 -2.5273948770524326 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74.jpg resize: (2160, 3264) 1357865873 1.8306089618664219 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70.jpg resize: (2160, 3264) 1357865805 -1.431510279548319 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37.jpg resize: (2160, 3264) 1357865723 -1.4245801165853418 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37.jpg resize: (2160, 3264) 1357865473 1.2580113398142698 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a.jpg resize: (2160, 3264) 1357863818 -1.4011696873615014 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817628_0.png resize: (505, 445) 1357974933 -1.6404457405519548 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817622_0.png resize: (141, 269) 1357974934 -2.416412378218831 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817625_0.png resize: (255, 770) 1357974935 -2.4581013918871046 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817621_0.png resize: (757, 763) 1357974936 -2.2621309223362465 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817623_0.png resize: (262, 455) 1357974937 -0.7021264621689705 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817634_0.png resize: (117, 159) 1357974938 -0.7937738408618724 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817633_0.png resize: (146, 178) 1357974939 -0.5102859433833676 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817632_0.png resize: (335, 266) 1357974940 -0.7892178454160929 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817644_0.png resize: (325, 311) 1357974941 -0.6814005953403812 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817647_0.png resize: (511, 329) 1357974942 -0.9747128358293204 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817642_0.png resize: (308, 223) 1357974943 -0.04825836481083796 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817640_0.png resize: (325, 320) 1357974944 -0.6515381682845616 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817646_0.png resize: (92, 121) 1357974945 -0.10755386546027138 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817637_0.png resize: (260, 250) 1357974946 -0.06835016002380667 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817638_0.png resize: (182, 267) 1357974947 -2.7059074942419827 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817635_0.png resize: (233, 233) 1357974948 -0.222618561355967 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817639_0.png resize: (133, 180) 1357974949 -2.8698225248231966 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817641_0.png resize: (337, 285) 1357974950 -1.5820675238478874 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817654_0.png resize: (103, 102) 1357974951 -0.08811806983560427 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817652_0.png resize: (242, 190) 1357974952 0.9583731920661491 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817655_0.png resize: (113, 221) 1357974953 0.9938029999660647 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817649_0.png resize: (182, 248) 1357974954 -0.9796945093090023 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817648_0.png resize: (270, 302) 1357974955 1.441571375211 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817656_0.png resize: (228, 142) 1357974956 -0.6923127434754925 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817657_0.png resize: (453, 75) 1357974957 -3.0329766426351785 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817663_0.png resize: (187, 178) 1357974958 0.4085948328857146 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817664_0.png resize: (172, 178) 1357974959 -1.3490643930422586 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817666_0.png resize: (165, 215) 1357974960 -0.4545091632112005 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817674_0.png resize: (112, 146) 1357974961 -0.9628842492241834 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817662_0.png resize: (277, 154) 1357974963 -1.3897087877958856 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817670_0.png resize: (244, 109) 1357974964 -0.46817394721380634 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817668_0.png resize: (243, 131) 1357974965 -1.9908881355591974 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817669_0.png resize: (167, 176) 1357974966 -2.386427156598118 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817661_0.png resize: (176, 135) 1357974967 -1.20633824760865 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817676_0.png resize: (184, 186) 1357974968 -1.1797338564727329 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817681_0.png resize: (175, 300) 1357974969 -0.7017005747127868 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817677_0.png resize: (174, 124) 1357974970 1.5377867867364443 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817685_0.png resize: (94, 96) 1357974971 0.9933315960666504 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817678_0.png resize: (148, 140) 1357974972 0.6259020337410545 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817683_0.png resize: (785, 317) 1357974973 -0.6809902492395876 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817675_0.png resize: (260, 232) 1357974974 -0.5623441961081456 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817689_0.png resize: (87, 166) 1357974975 -2.3930734549070842 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817686_0.png resize: (227, 192) 1357974976 -1.2555253718303603 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817690_0.png resize: (143, 97) 1357974978 -0.7347912592496432 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817697_0.png resize: (300, 250) 1357974979 -2.1334271446638873 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817701_0.png resize: (109, 176) 1357974980 -0.79933593855088 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817691_0.png resize: (211, 187) 1357974982 -2.680133203723461 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817704_0.png resize: (169, 157) 1357974983 -0.6840608305864966 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817692_0.png resize: (100, 248) 1357974984 -0.35555235265561347 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817699_0.png resize: (84, 134) 1357974986 -0.27962173525473333 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817696_0.png resize: (196, 496) 1357974987 -1.7373698232130057 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817698_0.png resize: (147, 180) 1357974988 -1.6485165412713751 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817705_0.png resize: (159, 471) 1357974990 -0.9390976916679331 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817687_0.png resize: (920, 694) 1357974991 -2.450704900290895 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817702_0.png resize: (149, 101) 1357974992 0.30305365530532397 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817693_0.png resize: (116, 214) 1357974994 -0.3333215498324062 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817709_0.png resize: (221, 109) 1357974995 -1.065987905925259 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817712_0.png resize: (435, 179) 1357974996 -2.657003861363224 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817718_0.png resize: (118, 122) 1357974998 -1.551599875425676 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817710_0.png resize: (407, 166) 1357974999 -1.9290621176437919 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817722_0.png resize: (470, 535) 1357975000 -1.169257863438872 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817708_0.png resize: (126, 279) 1357975001 -0.8304481897121393 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817719_0.png resize: (128, 129) 1357975004 0.11920010002916809 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817715_0.png resize: (165, 116) 1357975005 -1.739923722328571 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817716_0.png resize: (217, 110) 1357975006 -0.1350193705721717 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817713_0.png resize: (191, 211) 1357975008 -1.4451092765380613 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817721_0.png resize: (88, 66) 1357975009 -0.4200570740535068 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817717_0.png resize: (78, 160) 1357975010 -2.0490163811304902 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817706_0.png resize: (188, 276) 1357975012 -1.9446975420853359 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817723_0.png resize: (102, 248) 1357975013 -0.36810721408939345 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817725_0.png resize: (782, 840) 1357975014 -0.545331299538718 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817727_0.png resize: (528, 554) 1357975016 -0.6759929879596641 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817735_0.png resize: (163, 149) 1357975017 0.6775523137271716 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817733_0.png resize: (66, 146) 1357975018 0.5015540685831227 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817734_0.png resize: (795, 846) 1357975020 0.18841937019208993 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817729_0.png resize: (122, 114) 1357975021 -1.4854029744460961 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817732_0.png resize: (381, 257) 1357975022 -0.13079064072729424 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817730_0.png resize: (76, 224) 1357975024 -1.6018960018705426 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817724_0.png resize: (419, 328) 1357975025 -1.2273519904285095 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817739_0.png resize: (189, 186) 1357975026 -0.5390536454516646 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817754_0.png resize: (408, 198) 1357975028 -1.5124478707030133 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817747_0.png resize: (365, 264) 1357975029 -2.0075272414283263 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817740_0.png resize: (401, 229) 1357975030 -0.6074070241872126 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817756_0.png resize: (439, 312) 1357975032 -0.4991699815709321 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817741_0.png resize: (233, 215) 1357975033 -0.20320012083766759 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817749_0.png resize: (145, 113) 1357975034 -0.6672736798531416 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817742_0.png resize: (358, 385) 1357975035 -0.2033721423773607 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817745_0.png resize: (476, 250) 1357975037 -1.1894846009045386 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817751_0.png resize: (195, 213) 1357975038 -1.9621568574481976 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817738_0.png resize: (332, 183) 1357975039 -0.789839632541001 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817631_0.png resize: (125, 217) 1357975142 -1.2415677153662283 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817636_0.png resize: (146, 186) 1357975144 -2.565054758056852 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817659_0.png resize: (150, 236) 1357975146 -1.838112238883937 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817660_0.png resize: (98, 161) 1357975147 -1.2409885888826173 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817671_0.png resize: (166, 74) 1357975150 -1.411218140569119 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817703_0.png resize: (144, 108) 1357975151 -0.005534122199019497 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817688_0.png resize: (642, 668) 1357975152 -1.3800022400877374 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817707_0.png resize: (182, 156) 1357975155 0.06475987573034951 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817714_0.png resize: (149, 100) 1357975156 -1.0413509505594896 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817711_0.png resize: (176, 292) 1357975157 -0.8511813238510808 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817726_0.png resize: (166, 253) 1357975160 -0.861141764055579 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817737_0.png resize: (549, 441) 1357975161 -0.8417189862273663 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817728_0.png resize: (287, 383) 1357975163 -0.677236877563202 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817755_0.png resize: (92, 124) 1357975166 -0.23581215980351442 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817746_0.png resize: (485, 581) 1357975167 -2.353244690932756 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817667_0.png resize: (122, 174) 1357975194 -1.5084198270258091 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817629_0.png resize: (263, 535) 1357975376 -1.3059542462406193 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817630_0.png resize: (250, 218) 1357975378 -2.849271923491234 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817624_0.png resize: (192, 495) 1357975380 -1.6899549705493926 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817626_0.png resize: (262, 322) 1357975381 -0.1347453173488184 treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817627_0.png resize: (300, 304) 1357975383 -2.0810813524115894 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817645_0.png resize: (183, 348) 1357975385 -2.4446699303211457 treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817643_0.png resize: (151, 288) 1357975386 -1.652911728056177 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817650_0.png resize: (393, 342) 1357975388 -0.6047811524915179 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817651_0.png resize: (347, 473) 1357975390 -0.6202073290190604 treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817653_0.png resize: (126, 261) 1357975391 -0.3159695389324343 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817665_0.png resize: (287, 253) 1357975393 -2.9239781137090346 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817658_0.png resize: (192, 427) 1357975395 0.11507347091021068 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817673_0.png resize: (216, 463) 1357975396 -1.781058812593623 treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817672_0.png resize: (254, 141) 1357975398 -1.5726280195557316 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817684_0.png resize: (224, 468) 1357975399 -1.6170449132506344 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817682_0.png resize: (159, 161) 1357975401 -1.2225394683945783 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817679_0.png resize: (208, 351) 1357975404 -0.5462432487254479 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817700_0.png resize: (310, 184) 1357975405 -2.4592524092523096 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817695_0.png resize: (182, 259) 1357975407 -0.8195578392457512 treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817694_0.png resize: (519, 327) 1357975409 -4.691727883717517 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817731_0.png resize: (364, 202) 1357975410 -2.03096925217741 treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817736_0.png resize: (478, 912) 1357975412 -1.0408272564096734 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817744_0.png resize: (367, 500) 1357975414 -1.9625972943920948 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817748_0.png resize: (323, 320) 1357975415 -0.6802553755849784 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817750_0.png resize: (139, 251) 1357975417 -0.9614577186173415 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817743_0.png resize: (406, 347) 1357975420 -0.867730611617251 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817753_0.png resize: (314, 329) 1357975421 -2.4455716736541477 treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817680_0.png resize: (187, 190) 1357975443 -0.6128334285972492 treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817752_0.png resize: (139, 175) 1357975444 1.321236142039161 treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817720_0.png resize: (80, 112) 1357975459 -0.27496600553587375 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 145 time used for this insertion : 0.01922440528869629 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 145 time used for this insertion : 0.037508487701416016 save missing photos in datou_result : time spend for datou_step_exec : 37.79109334945679 time spend to save output : 0.062482595443725586 total time spend for step 6 : 37.85357594490051 step7:brightness Tue May 13 22:35:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76.jpg treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c.jpg treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc.jpg treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2.jpg treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74.jpg treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70.jpg treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37.jpg treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37.jpg treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a.jpg treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817628_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817622_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817625_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817621_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817623_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817634_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817633_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817632_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817644_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817647_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817642_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817640_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817646_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817637_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817638_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817635_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817639_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817641_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817654_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817652_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817655_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817649_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817648_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817656_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817657_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817663_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817664_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817666_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817674_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817662_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817670_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817668_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817669_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817661_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817676_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817681_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817677_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817685_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817678_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817683_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817675_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817689_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817686_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817690_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817697_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817701_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817691_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817704_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817692_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817699_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817696_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817698_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817705_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817687_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817702_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817693_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817709_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817712_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817718_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817710_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817722_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817708_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817719_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817715_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817716_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817713_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817721_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817717_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817706_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817723_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817725_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817727_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817735_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817733_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817734_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817729_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817732_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817730_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817724_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817739_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817754_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817747_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817740_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817756_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817741_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817749_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817742_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817745_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817751_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817738_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817631_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817636_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817659_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817660_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817671_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817703_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817688_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817707_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817714_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817711_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817726_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817737_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817728_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817755_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817746_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817667_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817629_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817630_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817624_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817626_0.png treat image : temp/1747168229_363352_1357866036_8f96e0e4f5b4f8528ae033355d6d0c76_rle_crop_3795817627_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817645_0.png treat image : temp/1747168229_363352_1357866030_312cb5e2decbe007dbcda27d16276a7c_rle_crop_3795817643_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817650_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817651_0.png treat image : temp/1747168229_363352_1357866024_9f811c3ff82497604745233eb6927bdc_rle_crop_3795817653_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817665_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817658_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817673_0.png treat image : temp/1747168229_363352_1357865892_58e6531893bb53980eb1b40492b48cc2_rle_crop_3795817672_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817684_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817682_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817679_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817700_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817695_0.png treat image : temp/1747168229_363352_1357865805_dd908f4832eaac1e41e7c6345a6aee70_rle_crop_3795817694_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817731_0.png treat image : temp/1747168229_363352_1357865473_1cc1e27fc497af085f7f67c10b210e37_rle_crop_3795817736_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817744_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817748_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817750_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817743_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817753_0.png treat image : temp/1747168229_363352_1357865873_4ce6a55396bda926944760a359bdba74_rle_crop_3795817680_0.png treat image : temp/1747168229_363352_1357863818_ecef87aa23bed415e7d631b61800c67a_rle_crop_3795817752_0.png treat image : temp/1747168229_363352_1357865723_63c6b8229872a5b21f96d9fbfc2f2f37_rle_crop_3795817720_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 145 time used for this insertion : 0.017506122589111328 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 145 time used for this insertion : 0.033055782318115234 save missing photos in datou_result : time spend for datou_step_exec : 9.592851877212524 time spend to save output : 0.055589914321899414 total time spend for step 7 : 9.648441791534424 step8:velours_tree Tue May 13 22:35:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2631645202636719 time spend to save output : 4.1961669921875e-05 total time spend for step 8 : 0.26320648193359375 step9:send_mail_cod Tue May 13 22:35:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22934121_13-05-2025_22_35_38.pdf 22937974 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette229379741747168538 22937975 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette229379751747168539 22937976 change filename to text .change filename to text .imagette229379761747168541 22937977 imagette229379771747168541 22937978 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette229379781747168541 22937980 change filename to text .imagette229379801747168542 22937981 imagette229379811747168542 22937982 imagette229379821747168542 22937983 change filename to text .imagette229379831747168542 22937984 imagette229379841747168542 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22934121 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22937974,22937975,22937976,22937977,22937978,22937979,22937980,22937981,22937982,22937983,22937984?tags=carton,pet_clair,autre,flou,papier,environnement,pet_fonce,background,pehd,metal,mal_croppe args[1357866036] : ((1357866036, -1.488323768867265, 492688767), (1357866036, -0.20598256022421388, 496442774), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357866030] : ((1357866030, -0.8332922896356084, 492688767), (1357866030, 0.938029392709135, 2107752395), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357866024] : ((1357866024, -1.4653933444232876, 492688767), (1357866024, 0.3719246968783641, 2107752395), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357865892] : ((1357865892, -2.5273948770524326, 492609224), (1357865892, -0.19980869120401273, 496442774), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357865873] : ((1357865873, 1.8306089618664219, 492688767), (1357865873, 0.24511726511693113, 2107752395), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357865805] : ((1357865805, -1.431510279548319, 492688767), (1357865805, -0.3716789289438998, 496442774), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357865723] : ((1357865723, -1.4245801165853418, 492688767), (1357865723, -0.034136528010205475, 2107752395), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357865473] : ((1357865473, 1.2580113398142698, 492688767), (1357865473, 0.0993932607436437, 2107752395), '0.11598221400185589') We are sending mail with results at report@fotonower.com args[1357863818] : ((1357863818, -1.4011696873615014, 492688767), (1357863818, 0.05031778911440702, 2107752395), '0.11598221400185589') We are sending mail with results at report@fotonower.com refus_total : 0.11598221400185589 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=22934121 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934121_13-05-2025_22_35_38.pdf results_Auto_P22934121_13-05-2025_22_35_38.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934121_13-05-2025_22_35_38.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','22934121','results_Auto_P22934121_13-05-2025_22_35_38.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934121_13-05-2025_22_35_38.pdf','pdf','','0.59','0.11598221400185589') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/22934121

https://www.fotonower.com/image?json=false&list_photos_id=1357866036
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1357866030
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1357866024
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1357865892
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1357865873
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.8306089618664219)
https://www.fotonower.com/image?json=false&list_photos_id=1357865805
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1357865723
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1357865473
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.2580113398142698)
https://www.fotonower.com/image?json=false&list_photos_id=1357863818
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 11.60%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/22937974?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/22937975?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/22937976?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/22937978?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/22937980?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/22937983?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934121_13-05-2025_22_35_38.pdf.

Lien vers velours :https://www.fotonower.com/velours/22937974,22937975,22937976,22937977,22937978,22937979,22937980,22937981,22937982,22937983,22937984?tags=carton,pet_clair,autre,flou,papier,environnement,pet_fonce,background,pehd,metal,mal_croppe.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 13 May 2025 20:35:45 GMT Content-Length: 0 Connection: close X-Message-Id: rYzArzltTRy2nBgtkLoKHw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1357866036, 1357866030, 1357866024, 1357865892, 1357865873, 1357865805, 1357865723, 1357865473, 1357863818] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866036', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866030', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866024', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865892', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865873', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865805', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865723', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865473', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357863818', None, None, None, None, None, '2843491') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.016366958618164062 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.2944176197052 time spend to save output : 0.01663494110107422 total time spend for step 9 : 7.311052560806274 step10:split_time_score Tue May 13 22:35:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('16', 1), ('17', 8)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 13052025 22934121 Nombre de photos uploadées : 9 / 23040 (0%) 13052025 22934121 Nombre de photos taguées (types de déchets): 0 / 9 (0%) 13052025 22934121 Nombre de photos taguées (volume) : 0 / 9 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.7220458984375e-06 ????????? elapsed_time : fill_and_build_computed_from_old_data 0.00047206878662109375 elapsed_time : insert_dashboard_record_day_entry 0.02247166633605957 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22915576 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22915102 order by id desc limit 1 Qualite : 0.10380308566327769 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22915104_13-05-2025_10_01_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22915104 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22915104 AND mptpi.`type`=3594 To do Qualite : 0.13293756666439724 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22915108_13-05-2025_09_48_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22915108 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22915108 AND mptpi.`type`=3594 To do Qualite : 0.10491669432529954 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934039_13-05-2025_22_14_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934039 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934039 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934071 order by id desc limit 1 Qualite : 0.13382412513616562 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934073_13-05-2025_22_09_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934073 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934073 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934083 order by id desc limit 1 Qualite : 0.11376972907827847 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934093_13-05-2025_22_00_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934093 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934093 AND mptpi.`type`=3594 To do Qualite : 0.10474872385296709 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934096_13-05-2025_21_48_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934096 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934096 AND mptpi.`type`=3594 To do Qualite : 0.11598221400185589 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934121_13-05-2025_22_35_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934121 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934121 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934123 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934165 order by id desc limit 1 Qualite : 0.09959119932695298 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934169_13-05-2025_21_17_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934169 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934169 AND mptpi.`type`=3594 To do Qualite : 0.0978150287892935 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22934171_13-05-2025_21_07_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22934171 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22934171 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'13052025': {'nb_upload': 9, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1357866036, 1357866030, 1357866024, 1357865892, 1357865873, 1357865805, 1357865723, 1357865473, 1357863818] Looping around the photos to save general results len do output : 1 /22934121Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866036', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866030', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357866024', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865892', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865873', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865805', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865723', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357865473', None, None, None, None, None, '2843491') ('3318', None, None, None, None, None, None, None, '2843491') ('3318', '22934121', '1357863818', None, None, None, None, None, '2843491') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.014505624771118164 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.946213722229004 time spend to save output : 0.014824867248535156 total time spend for step 10 : 5.961038589477539 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 9 set_done_treatment 128.31user 123.96system 5:27.15elapsed 77%CPU (0avgtext+0avgdata 4844812maxresident)k 2354112inputs+73640outputs (12084major+12089012minor)pagefaults 0swaps