python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 409337 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2929958'] with mtr_portfolio_ids : ['23231814'] and first list_photo_ids : [] new path : /proc/409337/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.2713382244110107 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed May 21 22:50:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-21 22:50:32.365600: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-21 22:50:32.391043: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-21 22:50:32.392897: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa80c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-21 22:50:32.392951: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-21 22:50:32.396300: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-21 22:50:32.516964: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x317d22f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-21 22:50:32.517010: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-21 22:50:32.518581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 22:50:32.519055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 22:50:32.522121: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 22:50:32.524904: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 22:50:32.525283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 22:50:32.527949: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 22:50:32.529273: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 22:50:32.534592: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 22:50:32.536258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 22:50:32.536354: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 22:50:32.537183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 22:50:32.537200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 22:50:32.537210: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 22:50:32.539059: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9879 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-21 22:50:32.810224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 22:50:32.810306: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 22:50:32.810327: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 22:50:32.810346: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 22:50:32.810364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 22:50:32.810382: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 22:50:32.810400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 22:50:32.810418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 22:50:32.812070: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 22:50:32.813470: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-21 22:50:32.813518: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-21 22:50:32.813537: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 22:50:32.813554: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-21 22:50:32.813571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-21 22:50:32.813589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-21 22:50:32.813605: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-21 22:50:32.813622: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-21 22:50:32.815212: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-21 22:50:32.815248: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-21 22:50:32.815259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-21 22:50:32.815269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-21 22:50:32.816665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9879 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-21 22:50:45.293463: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-21 22:50:45.494494: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 13.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 32 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 36 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 Detection mask done ! Trying to reset tf kernel 409667 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.002619028091430664 nb_pixel_total : 103094 time to create 1 rle with old method : 0.11150884628295898 length of segment : 353 time for calcul the mask position with numpy : 0.00024271011352539062 nb_pixel_total : 15078 time to create 1 rle with old method : 0.016762971878051758 length of segment : 109 time for calcul the mask position with numpy : 0.0002009868621826172 nb_pixel_total : 6869 time to create 1 rle with old method : 0.00798177719116211 length of segment : 123 time for calcul the mask position with numpy : 0.0001666545867919922 nb_pixel_total : 8077 time to create 1 rle with old method : 0.009464740753173828 length of segment : 89 time for calcul the mask position with numpy : 0.0005786418914794922 nb_pixel_total : 38024 time to create 1 rle with old method : 0.04229450225830078 length of segment : 227 time for calcul the mask position with numpy : 0.00044155120849609375 nb_pixel_total : 22043 time to create 1 rle with old method : 0.02475571632385254 length of segment : 193 time for calcul the mask position with numpy : 0.00029921531677246094 nb_pixel_total : 16145 time to create 1 rle with old method : 0.01837325096130371 length of segment : 167 time for calcul the mask position with numpy : 0.0002911090850830078 nb_pixel_total : 16194 time to create 1 rle with old method : 0.018766164779663086 length of segment : 131 time for calcul the mask position with numpy : 0.00034308433532714844 nb_pixel_total : 17001 time to create 1 rle with old method : 0.020010948181152344 length of segment : 141 time for calcul the mask position with numpy : 0.00047326087951660156 nb_pixel_total : 23712 time to create 1 rle with old method : 0.02756786346435547 length of segment : 163 time for calcul the mask position with numpy : 0.0003330707550048828 nb_pixel_total : 13085 time to create 1 rle with old method : 0.015519380569458008 length of segment : 151 time for calcul the mask position with numpy : 0.0005183219909667969 nb_pixel_total : 21403 time to create 1 rle with old method : 0.02435302734375 length of segment : 286 time for calcul the mask position with numpy : 0.00039958953857421875 nb_pixel_total : 23768 time to create 1 rle with old method : 0.02697014808654785 length of segment : 192 time for calcul the mask position with numpy : 0.00026035308837890625 nb_pixel_total : 12341 time to create 1 rle with old method : 0.014597892761230469 length of segment : 146 time for calcul the mask position with numpy : 0.00013065338134765625 nb_pixel_total : 7722 time to create 1 rle with old method : 0.008848905563354492 length of segment : 89 time for calcul the mask position with numpy : 0.0008487701416015625 nb_pixel_total : 62637 time to create 1 rle with old method : 0.06943082809448242 length of segment : 329 time for calcul the mask position with numpy : 0.0004596710205078125 nb_pixel_total : 31545 time to create 1 rle with old method : 0.03559684753417969 length of segment : 237 time for calcul the mask position with numpy : 0.0004839897155761719 nb_pixel_total : 25519 time to create 1 rle with old method : 0.030640840530395508 length of segment : 201 time for calcul the mask position with numpy : 0.005106925964355469 nb_pixel_total : 196304 time to create 1 rle with new method : 0.015344381332397461 length of segment : 518 time for calcul the mask position with numpy : 0.0011153221130371094 nb_pixel_total : 25414 time to create 1 rle with old method : 0.03911852836608887 length of segment : 226 time for calcul the mask position with numpy : 0.0004508495330810547 nb_pixel_total : 13173 time to create 1 rle with old method : 0.02185654640197754 length of segment : 119 time for calcul the mask position with numpy : 0.0004961490631103516 nb_pixel_total : 19083 time to create 1 rle with old method : 0.03147315979003906 length of segment : 146 time for calcul the mask position with numpy : 0.0004646778106689453 nb_pixel_total : 12761 time to create 1 rle with old method : 0.021766185760498047 length of segment : 99 time for calcul the mask position with numpy : 0.0004241466522216797 nb_pixel_total : 18380 time to create 1 rle with old method : 0.023899316787719727 length of segment : 206 time for calcul the mask position with numpy : 0.0004329681396484375 nb_pixel_total : 8482 time to create 1 rle with old method : 0.016445636749267578 length of segment : 100 time for calcul the mask position with numpy : 0.0003657341003417969 nb_pixel_total : 11844 time to create 1 rle with old method : 0.020496368408203125 length of segment : 127 time for calcul the mask position with numpy : 0.0011439323425292969 nb_pixel_total : 52777 time to create 1 rle with old method : 0.0792384147644043 length of segment : 209 time for calcul the mask position with numpy : 0.0058689117431640625 nb_pixel_total : 232440 time to create 1 rle with new method : 0.01602339744567871 length of segment : 1107 time for calcul the mask position with numpy : 0.0003097057342529297 nb_pixel_total : 14773 time to create 1 rle with old method : 0.016964435577392578 length of segment : 128 time for calcul the mask position with numpy : 0.02053380012512207 nb_pixel_total : 366701 time to create 1 rle with new method : 0.030529260635375977 length of segment : 796 time for calcul the mask position with numpy : 0.0005145072937011719 nb_pixel_total : 22943 time to create 1 rle with old method : 0.02667689323425293 length of segment : 243 time for calcul the mask position with numpy : 0.0012781620025634766 nb_pixel_total : 58888 time to create 1 rle with old method : 0.06775856018066406 length of segment : 302 time for calcul the mask position with numpy : 0.0006375312805175781 nb_pixel_total : 35841 time to create 1 rle with old method : 0.0461575984954834 length of segment : 271 time for calcul the mask position with numpy : 0.002635478973388672 nb_pixel_total : 132700 time to create 1 rle with old method : 0.17216753959655762 length of segment : 471 time for calcul the mask position with numpy : 0.0007169246673583984 nb_pixel_total : 39250 time to create 1 rle with old method : 0.0454862117767334 length of segment : 144 time for calcul the mask position with numpy : 0.00019884109497070312 nb_pixel_total : 7698 time to create 1 rle with old method : 0.00899362564086914 length of segment : 86 time for calcul the mask position with numpy : 0.00042629241943359375 nb_pixel_total : 19141 time to create 1 rle with old method : 0.024015426635742188 length of segment : 131 time for calcul the mask position with numpy : 0.00024700164794921875 nb_pixel_total : 12955 time to create 1 rle with old method : 0.01498723030090332 length of segment : 142 time for calcul the mask position with numpy : 0.0008378028869628906 nb_pixel_total : 15927 time to create 1 rle with old method : 0.020574331283569336 length of segment : 297 time for calcul the mask position with numpy : 0.003036022186279297 nb_pixel_total : 165563 time to create 1 rle with new method : 0.008281707763671875 length of segment : 365 time for calcul the mask position with numpy : 0.0012125968933105469 nb_pixel_total : 25212 time to create 1 rle with old method : 0.03016972541809082 length of segment : 170 time for calcul the mask position with numpy : 0.0008542537689208984 nb_pixel_total : 21778 time to create 1 rle with old method : 0.026253700256347656 length of segment : 144 time for calcul the mask position with numpy : 0.0012178421020507812 nb_pixel_total : 32657 time to create 1 rle with old method : 0.04013800621032715 length of segment : 208 time for calcul the mask position with numpy : 0.005704641342163086 nb_pixel_total : 148871 time to create 1 rle with old method : 0.16542816162109375 length of segment : 661 time for calcul the mask position with numpy : 0.00037288665771484375 nb_pixel_total : 14588 time to create 1 rle with old method : 0.01587963104248047 length of segment : 127 time for calcul the mask position with numpy : 0.0005693435668945312 nb_pixel_total : 16169 time to create 1 rle with old method : 0.018637418746948242 length of segment : 114 time for calcul the mask position with numpy : 0.0015583038330078125 nb_pixel_total : 26501 time to create 1 rle with old method : 0.029748201370239258 length of segment : 187 time for calcul the mask position with numpy : 0.003447294235229492 nb_pixel_total : 103245 time to create 1 rle with old method : 0.1119239330291748 length of segment : 290 time for calcul the mask position with numpy : 0.0004858970642089844 nb_pixel_total : 10963 time to create 1 rle with old method : 0.012379646301269531 length of segment : 148 time for calcul the mask position with numpy : 0.002303600311279297 nb_pixel_total : 90224 time to create 1 rle with old method : 0.09925317764282227 length of segment : 242 time for calcul the mask position with numpy : 0.0075910091400146484 nb_pixel_total : 212245 time to create 1 rle with new method : 0.012403726577758789 length of segment : 418 time for calcul the mask position with numpy : 0.0014348030090332031 nb_pixel_total : 39940 time to create 1 rle with old method : 0.046437978744506836 length of segment : 142 time for calcul the mask position with numpy : 0.0007448196411132812 nb_pixel_total : 26487 time to create 1 rle with old method : 0.029877424240112305 length of segment : 152 time for calcul the mask position with numpy : 0.0010471343994140625 nb_pixel_total : 19248 time to create 1 rle with old method : 0.020868301391601562 length of segment : 191 time for calcul the mask position with numpy : 0.0009136199951171875 nb_pixel_total : 31104 time to create 1 rle with old method : 0.03615403175354004 length of segment : 226 time for calcul the mask position with numpy : 0.0007448196411132812 nb_pixel_total : 21358 time to create 1 rle with old method : 0.023889780044555664 length of segment : 195 time for calcul the mask position with numpy : 0.0007264614105224609 nb_pixel_total : 21321 time to create 1 rle with old method : 0.02338433265686035 length of segment : 213 time for calcul the mask position with numpy : 0.0019714832305908203 nb_pixel_total : 45206 time to create 1 rle with old method : 0.05391287803649902 length of segment : 201 time for calcul the mask position with numpy : 0.002249002456665039 nb_pixel_total : 72851 time to create 1 rle with old method : 0.08107304573059082 length of segment : 299 time for calcul the mask position with numpy : 0.0008537769317626953 nb_pixel_total : 22300 time to create 1 rle with old method : 0.025498628616333008 length of segment : 248 time for calcul the mask position with numpy : 0.0006928443908691406 nb_pixel_total : 13201 time to create 1 rle with old method : 0.01629185676574707 length of segment : 191 time for calcul the mask position with numpy : 0.0004119873046875 nb_pixel_total : 9305 time to create 1 rle with old method : 0.010749578475952148 length of segment : 125 time for calcul the mask position with numpy : 0.0010187625885009766 nb_pixel_total : 15374 time to create 1 rle with old method : 0.017457246780395508 length of segment : 175 time for calcul the mask position with numpy : 0.0004189014434814453 nb_pixel_total : 11266 time to create 1 rle with old method : 0.013549089431762695 length of segment : 76 time for calcul the mask position with numpy : 0.0005893707275390625 nb_pixel_total : 11633 time to create 1 rle with old method : 0.014158964157104492 length of segment : 72 time for calcul the mask position with numpy : 0.0015554428100585938 nb_pixel_total : 39742 time to create 1 rle with old method : 0.046584367752075195 length of segment : 169 time for calcul the mask position with numpy : 0.0021600723266601562 nb_pixel_total : 58626 time to create 1 rle with old method : 0.06482625007629395 length of segment : 280 time for calcul the mask position with numpy : 0.0012316703796386719 nb_pixel_total : 25125 time to create 1 rle with old method : 0.02813577651977539 length of segment : 286 time for calcul the mask position with numpy : 0.0010242462158203125 nb_pixel_total : 8838 time to create 1 rle with old method : 0.010239124298095703 length of segment : 171 time for calcul the mask position with numpy : 0.0005271434783935547 nb_pixel_total : 13217 time to create 1 rle with old method : 0.015082597732543945 length of segment : 83 time for calcul the mask position with numpy : 0.001960277557373047 nb_pixel_total : 62792 time to create 1 rle with old method : 0.07102417945861816 length of segment : 292 time for calcul the mask position with numpy : 0.0006020069122314453 nb_pixel_total : 13807 time to create 1 rle with old method : 0.0208284854888916 length of segment : 145 time for calcul the mask position with numpy : 0.0006732940673828125 nb_pixel_total : 20249 time to create 1 rle with old method : 0.024448156356811523 length of segment : 169 time for calcul the mask position with numpy : 0.0012211799621582031 nb_pixel_total : 25070 time to create 1 rle with old method : 0.039992570877075195 length of segment : 300 time for calcul the mask position with numpy : 0.0006246566772460938 nb_pixel_total : 13288 time to create 1 rle with old method : 0.01672077178955078 length of segment : 183 time for calcul the mask position with numpy : 0.0002987384796142578 nb_pixel_total : 12223 time to create 1 rle with old method : 0.013950347900390625 length of segment : 146 time for calcul the mask position with numpy : 0.0007529258728027344 nb_pixel_total : 31122 time to create 1 rle with old method : 0.03481173515319824 length of segment : 188 time for calcul the mask position with numpy : 0.001149892807006836 nb_pixel_total : 31239 time to create 1 rle with old method : 0.0349116325378418 length of segment : 277 time for calcul the mask position with numpy : 0.007327556610107422 nb_pixel_total : 249918 time to create 1 rle with new method : 0.014097929000854492 length of segment : 882 time for calcul the mask position with numpy : 0.0011789798736572266 nb_pixel_total : 35442 time to create 1 rle with old method : 0.03977203369140625 length of segment : 354 time for calcul the mask position with numpy : 0.002148866653442383 nb_pixel_total : 68438 time to create 1 rle with old method : 0.07513189315795898 length of segment : 270 time for calcul the mask position with numpy : 0.008689403533935547 nb_pixel_total : 256976 time to create 1 rle with new method : 0.01772785186767578 length of segment : 614 time for calcul the mask position with numpy : 0.0011723041534423828 nb_pixel_total : 28446 time to create 1 rle with old method : 0.03180670738220215 length of segment : 302 time for calcul the mask position with numpy : 0.0007250308990478516 nb_pixel_total : 20220 time to create 1 rle with old method : 0.023380517959594727 length of segment : 140 time for calcul the mask position with numpy : 0.00211334228515625 nb_pixel_total : 67005 time to create 1 rle with old method : 0.07576751708984375 length of segment : 236 time for calcul the mask position with numpy : 0.0024552345275878906 nb_pixel_total : 60372 time to create 1 rle with old method : 0.06665682792663574 length of segment : 324 time for calcul the mask position with numpy : 0.002104520797729492 nb_pixel_total : 59357 time to create 1 rle with old method : 0.06430697441101074 length of segment : 427 time for calcul the mask position with numpy : 0.005432605743408203 nb_pixel_total : 158472 time to create 1 rle with new method : 0.011092901229858398 length of segment : 968 time for calcul the mask position with numpy : 0.0007147789001464844 nb_pixel_total : 11952 time to create 1 rle with old method : 0.014182806015014648 length of segment : 122 time for calcul the mask position with numpy : 0.0004475116729736328 nb_pixel_total : 14371 time to create 1 rle with old method : 0.017406463623046875 length of segment : 153 time for calcul the mask position with numpy : 0.0003037452697753906 nb_pixel_total : 5869 time to create 1 rle with old method : 0.007104635238647461 length of segment : 87 time for calcul the mask position with numpy : 0.0011823177337646484 nb_pixel_total : 34088 time to create 1 rle with old method : 0.04138064384460449 length of segment : 199 time for calcul the mask position with numpy : 0.0010118484497070312 nb_pixel_total : 39767 time to create 1 rle with old method : 0.04368948936462402 length of segment : 165 time for calcul the mask position with numpy : 0.001519918441772461 nb_pixel_total : 64466 time to create 1 rle with old method : 0.07180333137512207 length of segment : 270 time for calcul the mask position with numpy : 0.0015025138854980469 nb_pixel_total : 37217 time to create 1 rle with old method : 0.04091954231262207 length of segment : 340 time spent for convertir_results : 7.259227752685547 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 95 chid ids of type : 3594 Number RLEs to save : 23317 save missing photos in datou_result : time spend for datou_step_exec : 60.86388874053955 time spend to save output : 1.6291847229003906 total time spend for step 1 : 62.49307346343994 step2:crop_condition Wed May 21 22:51:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 95 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 56 About to insert : list_path_to_insert length 56 new photo from crops ! About to upload 56 photos upload in portfolio : 3736932 init cache_photo without model_param we have 56 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747860704_409337 we have uploaded 56 photos in the portfolio 3736932 time of upload the photos Elapsed time : 15.263877630233765 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3736932 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747860724_409337 we have uploaded 14 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.218290090560913 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747860728_409337 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.631697416305542 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 3736932 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747860736_409337 we have uploaded 20 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.497889041900635 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747860743_409337 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9739189147949219 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747860745_409337 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5638430118560791 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1360193010, 1360192824, 1360192782, 1360192384, 1360192353, 1360192320, 1360192290, 1360192256] Looping around the photos to save general results len do output : 95 /1360217765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360217996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360218031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360193010', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192824', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192782', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192384', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192353', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192320', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192290', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192256', None, None, None, None, None, '2929958') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 293 time used for this insertion : 0.02678084373474121 save_final save missing photos in datou_result : time spend for datou_step_exec : 53.140318155288696 time spend to save output : 0.030379056930541992 total time spend for step 2 : 53.17069721221924 step3:rle_unique_nms_with_priority Wed May 21 22:52:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 95 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 4.900386095046997 time for calcul the mask position with numpy : 0.6945686340332031 nb_pixel_total : 6713444 time to create 1 rle with new method : 0.9957692623138428 time for calcul the mask position with numpy : 0.033772945404052734 nb_pixel_total : 12341 time to create 1 rle with old method : 0.01993250846862793 time for calcul the mask position with numpy : 0.02379584312438965 nb_pixel_total : 23768 time to create 1 rle with old method : 0.0356597900390625 time for calcul the mask position with numpy : 0.02071523666381836 nb_pixel_total : 21365 time to create 1 rle with old method : 0.023224592208862305 time for calcul the mask position with numpy : 0.020978450775146484 nb_pixel_total : 13085 time to create 1 rle with old method : 0.01454782485961914 time for calcul the mask position with numpy : 0.02207159996032715 nb_pixel_total : 23712 time to create 1 rle with old method : 0.02930736541748047 time for calcul the mask position with numpy : 0.02244734764099121 nb_pixel_total : 17001 time to create 1 rle with old method : 0.019039392471313477 time for calcul the mask position with numpy : 0.02139592170715332 nb_pixel_total : 16194 time to create 1 rle with old method : 0.017708301544189453 time for calcul the mask position with numpy : 0.02095317840576172 nb_pixel_total : 16145 time to create 1 rle with old method : 0.01749134063720703 time for calcul the mask position with numpy : 0.02231764793395996 nb_pixel_total : 22043 time to create 1 rle with old method : 0.02592635154724121 time for calcul the mask position with numpy : 0.021996736526489258 nb_pixel_total : 38024 time to create 1 rle with old method : 0.04152846336364746 time for calcul the mask position with numpy : 0.021903038024902344 nb_pixel_total : 8077 time to create 1 rle with old method : 0.009072303771972656 time for calcul the mask position with numpy : 0.021426916122436523 nb_pixel_total : 6869 time to create 1 rle with old method : 0.00755620002746582 time for calcul the mask position with numpy : 0.021287202835083008 nb_pixel_total : 15078 time to create 1 rle with old method : 0.016559123992919922 time for calcul the mask position with numpy : 0.02199268341064453 nb_pixel_total : 103094 time to create 1 rle with old method : 0.11396455764770508 create new chi : 2.4323890209198 time to delete rle : 0.01825118064880371 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++++Number RLEs to save : 7096 TO DO : save crop sub photo not yet done ! save time : 0.5305807590484619 nb_obj : 12 nb_hashtags : 3 time to prepare the origin masks : 6.832636833190918 time for calcul the mask position with numpy : 0.724374532699585 nb_pixel_total : 6618604 time to create 1 rle with new method : 1.3173503875732422 time for calcul the mask position with numpy : 0.0213623046875 nb_pixel_total : 11844 time to create 1 rle with old method : 0.012821674346923828 time for calcul the mask position with numpy : 0.022125720977783203 nb_pixel_total : 8482 time to create 1 rle with old method : 0.009494781494140625 time for calcul the mask position with numpy : 0.033586978912353516 nb_pixel_total : 18380 time to create 1 rle with old method : 0.020180225372314453 time for calcul the mask position with numpy : 0.036298274993896484 nb_pixel_total : 12761 time to create 1 rle with old method : 0.014406681060791016 time for calcul the mask position with numpy : 0.03804779052734375 nb_pixel_total : 19083 time to create 1 rle with old method : 0.021185636520385742 time for calcul the mask position with numpy : 0.03502488136291504 nb_pixel_total : 13173 time to create 1 rle with old method : 0.01422262191772461 time for calcul the mask position with numpy : 0.02183842658996582 nb_pixel_total : 24186 time to create 1 rle with old method : 0.026708126068115234 time for calcul the mask position with numpy : 0.024506568908691406 nb_pixel_total : 196304 time to create 1 rle with new method : 1.2530291080474854 time for calcul the mask position with numpy : 0.022292137145996094 nb_pixel_total : 25519 time to create 1 rle with old method : 0.027977943420410156 time for calcul the mask position with numpy : 0.022820234298706055 nb_pixel_total : 31545 time to create 1 rle with old method : 0.03455710411071777 time for calcul the mask position with numpy : 0.027265548706054688 nb_pixel_total : 62637 time to create 1 rle with old method : 0.0670328140258789 time for calcul the mask position with numpy : 0.022936105728149414 nb_pixel_total : 7722 time to create 1 rle with old method : 0.008470773696899414 create new chi : 3.943225145339966 time to delete rle : 0.0011506080627441406 batch 1 Loaded 25 chid ids of type : 3594 ++++++++++++Number RLEs to save : 6863 TO DO : save crop sub photo not yet done ! save time : 0.4623076915740967 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 7.048916578292847 time for calcul the mask position with numpy : 0.4741020202636719 nb_pixel_total : 6054133 time to create 1 rle with new method : 0.8617227077484131 time for calcul the mask position with numpy : 0.0319979190826416 nb_pixel_total : 12955 time to create 1 rle with old method : 0.014161109924316406 time for calcul the mask position with numpy : 0.020429611206054688 nb_pixel_total : 19141 time to create 1 rle with old method : 0.021094322204589844 time for calcul the mask position with numpy : 0.020349502563476562 nb_pixel_total : 7698 time to create 1 rle with old method : 0.008486032485961914 time for calcul the mask position with numpy : 0.02021646499633789 nb_pixel_total : 39250 time to create 1 rle with old method : 0.04218912124633789 time for calcul the mask position with numpy : 0.023299217224121094 nb_pixel_total : 132700 time to create 1 rle with old method : 0.14436721801757812 time for calcul the mask position with numpy : 0.02150440216064453 nb_pixel_total : 35841 time to create 1 rle with old method : 0.03949570655822754 time for calcul the mask position with numpy : 0.02142953872680664 nb_pixel_total : 58888 time to create 1 rle with old method : 0.06494617462158203 time for calcul the mask position with numpy : 0.020861148834228516 nb_pixel_total : 22943 time to create 1 rle with old method : 0.025292396545410156 time for calcul the mask position with numpy : 0.022460222244262695 nb_pixel_total : 366701 time to create 1 rle with new method : 0.424069881439209 time for calcul the mask position with numpy : 0.022261381149291992 nb_pixel_total : 14773 time to create 1 rle with old method : 0.016310453414916992 time for calcul the mask position with numpy : 0.023020505905151367 nb_pixel_total : 232440 time to create 1 rle with new method : 0.9554133415222168 time for calcul the mask position with numpy : 0.02084636688232422 nb_pixel_total : 52777 time to create 1 rle with old method : 0.05537843704223633 create new chi : 3.494868755340576 time to delete rle : 0.0013074874877929688 batch 1 Loaded 25 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 10219 TO DO : save crop sub photo not yet done ! save time : 0.6404366493225098 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 9.036755800247192 time for calcul the mask position with numpy : 0.695641279220581 nb_pixel_total : 6049518 time to create 1 rle with new method : 1.0228004455566406 time for calcul the mask position with numpy : 0.021193504333496094 nb_pixel_total : 31104 time to create 1 rle with old method : 0.03431844711303711 time for calcul the mask position with numpy : 0.02268385887145996 nb_pixel_total : 19248 time to create 1 rle with old method : 0.02109694480895996 time for calcul the mask position with numpy : 0.021957874298095703 nb_pixel_total : 26487 time to create 1 rle with old method : 0.02833271026611328 time for calcul the mask position with numpy : 0.022200584411621094 nb_pixel_total : 39940 time to create 1 rle with old method : 0.0434422492980957 time for calcul the mask position with numpy : 0.0226442813873291 nb_pixel_total : 212245 time to create 1 rle with new method : 0.7866191864013672 time for calcul the mask position with numpy : 0.025162458419799805 nb_pixel_total : 90224 time to create 1 rle with old method : 0.09830904006958008 time for calcul the mask position with numpy : 0.021117448806762695 nb_pixel_total : 10963 time to create 1 rle with old method : 0.01150655746459961 time for calcul the mask position with numpy : 0.02094745635986328 nb_pixel_total : 103245 time to create 1 rle with old method : 0.1105189323425293 time for calcul the mask position with numpy : 0.022402524948120117 nb_pixel_total : 26501 time to create 1 rle with old method : 0.028806447982788086 time for calcul the mask position with numpy : 0.02243208885192871 nb_pixel_total : 16169 time to create 1 rle with old method : 0.01777505874633789 time for calcul the mask position with numpy : 0.021694660186767578 nb_pixel_total : 14588 time to create 1 rle with old method : 0.015653610229492188 time for calcul the mask position with numpy : 0.022076129913330078 nb_pixel_total : 148871 time to create 1 rle with old method : 0.15683341026306152 time for calcul the mask position with numpy : 0.020023822784423828 nb_pixel_total : 32657 time to create 1 rle with old method : 0.03434348106384277 time for calcul the mask position with numpy : 0.020040273666381836 nb_pixel_total : 21778 time to create 1 rle with old method : 0.022450685501098633 time for calcul the mask position with numpy : 0.020041227340698242 nb_pixel_total : 25212 time to create 1 rle with old method : 0.026482820510864258 time for calcul the mask position with numpy : 0.02162909507751465 nb_pixel_total : 165563 time to create 1 rle with new method : 0.6425492763519287 time for calcul the mask position with numpy : 0.022241592407226562 nb_pixel_total : 15927 time to create 1 rle with old method : 0.017343521118164062 create new chi : 4.274785041809082 time to delete rle : 0.0014421939849853516 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 10324 TO DO : save crop sub photo not yet done ! save time : 0.6686835289001465 nb_obj : 15 nb_hashtags : 3 time to prepare the origin masks : 5.827395915985107 time for calcul the mask position with numpy : 0.5934534072875977 nb_pixel_total : 6664849 time to create 1 rle with new method : 0.7600677013397217 time for calcul the mask position with numpy : 0.019150495529174805 nb_pixel_total : 13217 time to create 1 rle with old method : 0.013228178024291992 time for calcul the mask position with numpy : 0.019437074661254883 nb_pixel_total : 4866 time to create 1 rle with old method : 0.005041599273681641 time for calcul the mask position with numpy : 0.020479679107666016 nb_pixel_total : 25125 time to create 1 rle with old method : 0.026127099990844727 time for calcul the mask position with numpy : 0.021352767944335938 nb_pixel_total : 58626 time to create 1 rle with old method : 0.059879302978515625 time for calcul the mask position with numpy : 0.020130634307861328 nb_pixel_total : 39742 time to create 1 rle with old method : 0.0402674674987793 time for calcul the mask position with numpy : 0.019791603088378906 nb_pixel_total : 11633 time to create 1 rle with old method : 0.011925697326660156 time for calcul the mask position with numpy : 0.019521236419677734 nb_pixel_total : 11266 time to create 1 rle with old method : 0.011649370193481445 time for calcul the mask position with numpy : 0.02056717872619629 nb_pixel_total : 15374 time to create 1 rle with old method : 0.015481233596801758 time for calcul the mask position with numpy : 0.019971847534179688 nb_pixel_total : 9305 time to create 1 rle with old method : 0.009499549865722656 time for calcul the mask position with numpy : 0.020790576934814453 nb_pixel_total : 13201 time to create 1 rle with old method : 0.01361989974975586 time for calcul the mask position with numpy : 0.020579814910888672 nb_pixel_total : 22300 time to create 1 rle with old method : 0.023583412170410156 time for calcul the mask position with numpy : 0.020848989486694336 nb_pixel_total : 72851 time to create 1 rle with old method : 0.07495594024658203 time for calcul the mask position with numpy : 0.020610809326171875 nb_pixel_total : 45206 time to create 1 rle with old method : 0.047223567962646484 time for calcul the mask position with numpy : 0.021476030349731445 nb_pixel_total : 21321 time to create 1 rle with old method : 0.0221555233001709 time for calcul the mask position with numpy : 0.020145893096923828 nb_pixel_total : 21358 time to create 1 rle with old method : 0.022150754928588867 create new chi : 2.0903115272521973 time to delete rle : 0.001055002212524414 batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 7653 TO DO : save crop sub photo not yet done ! save time : 0.5127472877502441 nb_obj : 10 nb_hashtags : 4 time to prepare the origin masks : 4.633752107620239 time for calcul the mask position with numpy : 0.40060877799987793 nb_pixel_total : 6555090 time to create 1 rle with new method : 0.7932755947113037 time for calcul the mask position with numpy : 0.021311521530151367 nb_pixel_total : 35442 time to create 1 rle with old method : 0.03829216957092285 time for calcul the mask position with numpy : 0.022620439529418945 nb_pixel_total : 249918 time to create 1 rle with new method : 0.8635752201080322 time for calcul the mask position with numpy : 0.021515607833862305 nb_pixel_total : 31239 time to create 1 rle with old method : 0.0339207649230957 time for calcul the mask position with numpy : 0.021270275115966797 nb_pixel_total : 31122 time to create 1 rle with old method : 0.03360342979431152 time for calcul the mask position with numpy : 0.021703243255615234 nb_pixel_total : 12223 time to create 1 rle with old method : 0.01340031623840332 time for calcul the mask position with numpy : 0.02064990997314453 nb_pixel_total : 13288 time to create 1 rle with old method : 0.014480829238891602 time for calcul the mask position with numpy : 0.021567821502685547 nb_pixel_total : 25070 time to create 1 rle with old method : 0.02710890769958496 time for calcul the mask position with numpy : 0.021252870559692383 nb_pixel_total : 20249 time to create 1 rle with old method : 0.02185797691345215 time for calcul the mask position with numpy : 0.022091150283813477 nb_pixel_total : 13807 time to create 1 rle with old method : 0.014799118041992188 time for calcul the mask position with numpy : 0.03387856483459473 nb_pixel_total : 62792 time to create 1 rle with old method : 0.06815743446350098 create new chi : 2.613142728805542 time to delete rle : 0.0012066364288330078 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 8032 TO DO : save crop sub photo not yet done ! save time : 0.5327634811401367 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 6.117812156677246 time for calcul the mask position with numpy : 0.6880810260772705 nb_pixel_total : 6304631 time to create 1 rle with new method : 0.6926271915435791 time for calcul the mask position with numpy : 0.020760774612426758 nb_pixel_total : 14371 time to create 1 rle with old method : 0.015490055084228516 time for calcul the mask position with numpy : 0.02257823944091797 nb_pixel_total : 11952 time to create 1 rle with old method : 0.019140243530273438 time for calcul the mask position with numpy : 0.024252891540527344 nb_pixel_total : 158472 time to create 1 rle with new method : 0.6499800682067871 time for calcul the mask position with numpy : 0.02209329605102539 nb_pixel_total : 59357 time to create 1 rle with old method : 0.06729483604431152 time for calcul the mask position with numpy : 0.020905017852783203 nb_pixel_total : 60372 time to create 1 rle with old method : 0.06506562232971191 time for calcul the mask position with numpy : 0.02421879768371582 nb_pixel_total : 67005 time to create 1 rle with old method : 0.07168078422546387 time for calcul the mask position with numpy : 0.03316450119018555 nb_pixel_total : 20220 time to create 1 rle with old method : 0.021359682083129883 time for calcul the mask position with numpy : 0.033036231994628906 nb_pixel_total : 28446 time to create 1 rle with old method : 0.03106689453125 time for calcul the mask position with numpy : 0.03639078140258789 nb_pixel_total : 256976 time to create 1 rle with new method : 0.7664515972137451 time for calcul the mask position with numpy : 0.033768415451049805 nb_pixel_total : 68438 time to create 1 rle with old method : 0.07407522201538086 create new chi : 3.5128068923950195 time to delete rle : 0.00121307373046875 batch 1 Loaded 21 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 9272 TO DO : save crop sub photo not yet done ! save time : 0.5739579200744629 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 2.2934563159942627 time for calcul the mask position with numpy : 0.4453883171081543 nb_pixel_total : 6868833 time to create 1 rle with new method : 0.906381368637085 time for calcul the mask position with numpy : 0.019988059997558594 nb_pixel_total : 37217 time to create 1 rle with old method : 0.04011249542236328 time for calcul the mask position with numpy : 0.020305871963500977 nb_pixel_total : 64466 time to create 1 rle with old method : 0.06910157203674316 time for calcul the mask position with numpy : 0.02095937728881836 nb_pixel_total : 39767 time to create 1 rle with old method : 0.04165077209472656 time for calcul the mask position with numpy : 0.019987106323242188 nb_pixel_total : 34088 time to create 1 rle with old method : 0.03491377830505371 time for calcul the mask position with numpy : 0.02009272575378418 nb_pixel_total : 5869 time to create 1 rle with old method : 0.006089687347412109 create new chi : 1.6808907985687256 time to delete rle : 0.0006167888641357422 batch 1 Loaded 11 chid ids of type : 3594 +++++++++Number RLEs to save : 4282 TO DO : save crop sub photo not yet done ! save time : 0.27585506439208984 map_output_result : {1360193010: (0.0, 'Should be the crop_list due to order', 0), 1360192824: (0.0, 'Should be the crop_list due to order', 0), 1360192782: (0.0, 'Should be the crop_list due to order', 0), 1360192384: (0.0, 'Should be the crop_list due to order', 0), 1360192353: (0.0, 'Should be the crop_list due to order', 0), 1360192320: (0.0, 'Should be the crop_list due to order', 0), 1360192290: (0.0, 'Should be the crop_list due to order', 0), 1360192256: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1360193010, 1360192824, 1360192782, 1360192384, 1360192353, 1360192320, 1360192290, 1360192256] Looping around the photos to save general results len do output : 8 /1360193010.Didn't retrieve data . /1360192824.Didn't retrieve data . /1360192782.Didn't retrieve data . /1360192384.Didn't retrieve data . /1360192353.Didn't retrieve data . /1360192320.Didn't retrieve data . /1360192290.Didn't retrieve data . /1360192256.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360193010', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192824', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192782', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192384', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192353', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192320', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192290', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192256', None, None, None, None, None, '2929958') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.01328587532043457 save_final save missing photos in datou_result : time spend for datou_step_exec : 76.50232195854187 time spend to save output : 0.013915777206420898 total time spend for step 3 : 76.51623773574829 step4:ventilate_hashtags_in_portfolio Wed May 21 22:53:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23231814 get user id for portfolio 23231814 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23231814 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','pet_fonce','papier','flou','background','autre','pet_clair','carton','environnement','mal_croppe','pehd')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23231814 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','pet_fonce','papier','flou','background','autre','pet_clair','carton','environnement','mal_croppe','pehd')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23231814 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','pet_fonce','papier','flou','background','autre','pet_clair','carton','environnement','mal_croppe','pehd')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23233051,23233052,23233053,23233054,23233055,23233056,23233057,23233058,23233059,23233060,23233061?tags=metal,pet_fonce,papier,flou,background,autre,pet_clair,carton,environnement,mal_croppe,pehd Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1360193010, 1360192824, 1360192782, 1360192384, 1360192353, 1360192320, 1360192290, 1360192256] Looping around the photos to save general results len do output : 1 /23231814. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360193010', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192824', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192782', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192384', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192353', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192320', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192290', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192256', None, None, None, None, None, '2929958') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.012847423553466797 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5683913230895996 time spend to save output : 0.012986898422241211 total time spend for step 4 : 1.5813782215118408 step5:final Wed May 21 22:53:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1360193010: ('0.08107557331381626',), 1360192824: ('0.08107557331381626',), 1360192782: ('0.08107557331381626',), 1360192384: ('0.08107557331381626',), 1360192353: ('0.08107557331381626',), 1360192320: ('0.08107557331381626',), 1360192290: ('0.08107557331381626',), 1360192256: ('0.08107557331381626',)} new output for save of step final : {1360193010: ('0.08107557331381626',), 1360192824: ('0.08107557331381626',), 1360192782: ('0.08107557331381626',), 1360192384: ('0.08107557331381626',), 1360192353: ('0.08107557331381626',), 1360192320: ('0.08107557331381626',), 1360192290: ('0.08107557331381626',), 1360192256: ('0.08107557331381626',)} [1360193010, 1360192824, 1360192782, 1360192384, 1360192353, 1360192320, 1360192290, 1360192256] Looping around the photos to save general results len do output : 8 /1360193010.Didn't retrieve data . /1360192824.Didn't retrieve data . /1360192782.Didn't retrieve data . /1360192384.Didn't retrieve data . /1360192353.Didn't retrieve data . /1360192320.Didn't retrieve data . /1360192290.Didn't retrieve data . /1360192256.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360193010', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192824', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192782', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192384', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192353', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192320', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192290', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192256', None, None, None, None, None, '2929958') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.014217376708984375 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12557005882263184 time spend to save output : 0.014775753021240234 total time spend for step 5 : 0.14034581184387207 step6:blur_detection Wed May 21 22:53:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891.jpg resize: (2160, 3264) 1360193010 -1.5958109666265803 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b.jpg resize: (2160, 3264) 1360192824 -0.5145555334314782 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9.jpg resize: (2160, 3264) 1360192782 2.5098748969272844 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572.jpg resize: (2160, 3264) 1360192384 -1.7259155497343688 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1.jpg resize: (2160, 3264) 1360192353 -2.9224855732334487 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f.jpg resize: (2160, 3264) 1360192320 -2.768958503016805 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e.jpg resize: (2160, 3264) 1360192290 -1.1381338548989424 treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c.jpg resize: (2160, 3264) 1360192256 1.0568974376182032 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578521_0.png resize: (348, 440) 1360217765 -1.8602434974132624 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578522_0.png resize: (109, 173) 1360217766 0.18654968515875076 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578523_0.png resize: (128, 145) 1360217767 -2.0174974205761584 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578524_0.png resize: (84, 146) 1360217768 -1.800564079688703 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578526_0.png resize: (167, 242) 1360217769 -1.263987224807438 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578528_0.png resize: (130, 202) 1360217770 -0.8024609640560358 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578529_0.png resize: (138, 204) 1360217771 -0.29852652954305253 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578530_0.png resize: (159, 205) 1360217772 -0.44684528253475997 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578531_0.png resize: (131, 171) 1360217773 -1.1943095842873412 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578532_0.png resize: (286, 197) 1360217774 -1.413289039940254 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578535_0.png resize: (89, 105) 1360217775 0.8395957725742114 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578536_0.png resize: (325, 285) 1360217777 -2.4467282908118637 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578537_0.png resize: (188, 231) 1360217778 0.21369172621771415 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578538_0.png resize: (194, 165) 1360217779 -0.27272520382040877 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578539_0.png resize: (518, 667) 1360217780 -0.9923473620443806 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578540_0.png resize: (212, 242) 1360217781 -0.8791706358863544 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578541_0.png resize: (119, 158) 1360217782 -0.5831354472140724 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578542_0.png resize: (140, 157) 1360217783 -0.7319464923791944 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578543_0.png resize: (99, 217) 1360217784 -0.8769571440487182 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578548_0.png resize: (797, 387) 1360217785 -1.7980529733069452 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578549_0.png resize: (128, 136) 1360217786 1.150680065017863 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578551_0.png resize: (243, 113) 1360217787 1.197516320055599 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578552_0.png resize: (252, 398) 1360217788 -0.02451471077916623 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578553_0.png resize: (217, 187) 1360217789 -0.4634624670141074 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578556_0.png resize: (84, 100) 1360217790 20.0 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578558_0.png resize: (142, 120) 1360217791 -1.284455003093056 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578559_0.png resize: (276, 218) 1360217792 -1.7103769443144752 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578561_0.png resize: (169, 271) 1360217793 -1.326173704898929 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578563_0.png resize: (198, 257) 1360217794 -0.6221141705783816 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578564_0.png resize: (530, 440) 1360217795 -1.1349711001804557 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578565_0.png resize: (108, 189) 1360217796 -1.3813424638555512 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578566_0.png resize: (110, 209) 1360217797 -0.0805173190504014 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578569_0.png resize: (148, 91) 1360217798 0.02231307949383967 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578572_0.png resize: (130, 418) 1360217799 -1.2534364547923498 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578577_0.png resize: (206, 124) 1360217800 -0.7809808044680417 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578580_0.png resize: (248, 121) 1360217801 -0.8813794892144171 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578581_0.png resize: (191, 135) 1360217802 -0.17808658220889692 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578582_0.png resize: (120, 111) 1360217803 -0.5971153070962907 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578583_0.png resize: (175, 202) 1360217804 -2.045501950441335 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578585_0.png resize: (72, 250) 1360217805 -1.6732215733740128 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578586_0.png resize: (165, 312) 1360217806 -2.1098783133090304 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578589_0.png resize: (193, 154) 1360217807 -1.1609258101391198 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578592_0.png resize: (144, 147) 1360217808 -1.0755845384808278 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578593_0.png resize: (169, 154) 1360217809 -0.5115274209046169 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578594_0.png resize: (299, 128) 1360217810 -1.2041331010999727 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578596_0.png resize: (146, 104) 1360217811 0.5817138996179505 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578598_0.png resize: (262, 190) 1360217812 -1.3089642081406865 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578601_0.png resize: (252, 412) 1360217813 0.06237386643951068 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578602_0.png resize: (605, 536) 1360217814 -1.6624974281484555 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578603_0.png resize: (286, 130) 1360217815 -0.5773597451677251 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578604_0.png resize: (140, 211) 1360217816 -0.034507392625345946 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578605_0.png resize: (229, 431) 1360217817 -1.0194196475618524 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578610_0.png resize: (145, 187) 1360217818 -2.3171409001461702 treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578611_0.png resize: (85, 84) 1360217819 3.165266371637166 treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578614_0.png resize: (227, 347) 1360217820 -0.6575473738675645 treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578615_0.png resize: (297, 346) 1360217821 -0.4605763938630338 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578527_0.png resize: (162, 147) 1360217839 -0.982108208301119 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578533_0.png resize: (192, 175) 1360217840 -0.009505856797642008 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578534_0.png resize: (146, 155) 1360217842 -1.2173384787962627 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578544_0.png resize: (206, 121) 1360217843 -0.8540974211052902 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578545_0.png resize: (100, 120) 1360217844 -0.7438269074507914 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578550_0.png resize: (728, 706) 1360217845 -0.3114587541065131 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578560_0.png resize: (327, 699) 1360217847 -0.8565634495801369 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578562_0.png resize: (143, 190) 1360217848 0.4029247637615033 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578574_0.png resize: (187, 213) 1360217849 -1.065419411775393 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578576_0.png resize: (187, 168) 1360217851 -0.8273777307573382 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578578_0.png resize: (192, 348) 1360217852 -0.9973168538936763 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578588_0.png resize: (286, 146) 1360217853 -1.431769934481402 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578591_0.png resize: (287, 309) 1360217855 -0.608428819372972 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578595_0.png resize: (183, 84) 1360217856 1.2840636247080355 treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578546_0.png resize: (125, 125) 1360217867 -1.8412043786082428 treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578525_0.png resize: (226, 233) 1360217922 -1.4344544118271052 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578547_0.png resize: (209, 337) 1360217924 -1.2160267689606787 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578554_0.png resize: (415, 552) 1360217926 0.29310526053285013 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578555_0.png resize: (140, 381) 1360217927 0.21373463183114894 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578567_0.png resize: (186, 224) 1360217929 -2.2670917800808144 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578568_0.png resize: (286, 435) 1360217930 -1.8314890702938722 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578570_0.png resize: (232, 512) 1360217932 -1.3066458862075965 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578571_0.png resize: (354, 757) 1360217933 -1.5098889716138832 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578575_0.png resize: (221, 182) 1360217935 -0.09366970013396958 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578579_0.png resize: (295, 298) 1360217937 -0.8253047397888282 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578584_0.png resize: (76, 172) 1360217938 -0.8024692897954517 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578587_0.png resize: (277, 290) 1360217941 -1.8961721550219437 treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578590_0.png resize: (83, 208) 1360217942 -1.6247249086704456 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578599_0.png resize: (657, 517) 1360217944 -3.3432287220642776 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578600_0.png resize: (354, 143) 1360217945 -1.10844126267596 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578606_0.png resize: (279, 434) 1360217947 -2.111150479745311 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578607_0.png resize: (329, 273) 1360217949 -1.9300262284183225 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578608_0.png resize: (603, 483) 1360217950 -0.739674380168445 treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578612_0.png resize: (199, 230) 1360217952 -0.9360888022197449 treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578613_0.png resize: (153, 336) 1360217953 -0.6236553692721502 treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578557_0.png resize: (163, 232) 1360217992 -1.5355533606475444 treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578597_0.png resize: (186, 216) 1360217994 -0.5963467710713183 treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578609_0.png resize: (122, 135) 1360217996 -0.06456833683848427 treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578573_0.png resize: (146, 232) 1360218031 -0.8914500247912043 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 103 time used for this insertion : 0.017986297607421875 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 103 time used for this insertion : 0.027830839157104492 save missing photos in datou_result : time spend for datou_step_exec : 26.737419605255127 time spend to save output : 0.05505514144897461 total time spend for step 6 : 26.7924747467041 step7:brightness Wed May 21 22:54:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891.jpg treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b.jpg treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9.jpg treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572.jpg treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1.jpg treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f.jpg treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e.jpg treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c.jpg treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578521_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578522_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578523_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578524_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578526_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578528_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578529_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578530_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578531_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578532_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578535_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578536_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578537_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578538_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578539_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578540_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578541_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578542_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578543_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578548_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578549_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578551_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578552_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578553_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578556_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578558_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578559_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578561_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578563_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578564_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578565_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578566_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578569_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578572_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578577_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578580_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578581_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578582_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578583_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578585_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578586_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578589_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578592_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578593_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578594_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578596_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578598_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578601_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578602_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578603_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578604_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578605_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578610_0.png treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578611_0.png treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578614_0.png treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578615_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578527_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578533_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578534_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578544_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578545_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578550_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578560_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578562_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578574_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578576_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578578_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578588_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578591_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578595_0.png treat image : temp/1747860628_409337_1360192824_084da0990f701911d22235ca2a97dd9b_rle_crop_3808578546_0.png treat image : temp/1747860628_409337_1360193010_c3d69ecc85a496cc321edba4ea072891_rle_crop_3808578525_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578547_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578554_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578555_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578567_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578568_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578570_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578571_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578575_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578579_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578584_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578587_0.png treat image : temp/1747860628_409337_1360192353_bb37c74363ff42e5e74b0f1f3de1d4a1_rle_crop_3808578590_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578599_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578600_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578606_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578607_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578608_0.png treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578612_0.png treat image : temp/1747860628_409337_1360192256_f5c2e397bd3d3e6ae872e03adcce934c_rle_crop_3808578613_0.png treat image : temp/1747860628_409337_1360192782_11a1f4801a74d90b4536b73ccabec0f9_rle_crop_3808578557_0.png treat image : temp/1747860628_409337_1360192320_fb290ec1223f2c8f2ac9f9bfa0fc915f_rle_crop_3808578597_0.png treat image : temp/1747860628_409337_1360192290_23af5b354d45471668083cfce956f76e_rle_crop_3808578609_0.png treat image : temp/1747860628_409337_1360192384_5fa13add7977f2cd13f6680fb7734572_rle_crop_3808578573_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 103 time used for this insertion : 0.01636219024658203 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 103 time used for this insertion : 0.030356884002685547 save missing photos in datou_result : time spend for datou_step_exec : 6.998953104019165 time spend to save output : 0.051363229751586914 total time spend for step 7 : 7.050316333770752 step8:velours_tree Wed May 21 22:54:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2901489734649658 time spend to save output : 6.389617919921875e-05 total time spend for step 8 : 0.29021286964416504 step9:send_mail_cod Wed May 21 22:54:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23231814_21-05-2025_22_54_17.pdf 23233051 change filename to text .imagette232330511747860857 23233052 change filename to text .imagette232330521747860857 23233053 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232330531747860857 23233054 imagette232330541747860859 23233055 imagette232330551747860859 23233056 change filename to text .change filename to text .change filename to text .imagette232330561747860859 23233057 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232330571747860859 23233058 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232330581747860861 23233060 imagette232330601747860862 23233061 imagette232330611747860862 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23231814 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23233051,23233052,23233053,23233054,23233055,23233056,23233057,23233058,23233059,23233060,23233061?tags=metal,pet_fonce,papier,flou,background,autre,pet_clair,carton,environnement,mal_croppe,pehd args[1360193010] : ((1360193010, -1.5958109666265803, 492688767), (1360193010, 0.16867252737838928, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192824] : ((1360192824, -0.5145555334314782, 492688767), (1360192824, 0.1967463649004122, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192782] : ((1360192782, 2.5098748969272844, 492688767), (1360192782, -0.026379496639864823, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192384] : ((1360192384, -1.7259155497343688, 492688767), (1360192384, 0.19444247380038457, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192353] : ((1360192353, -2.9224855732334487, 492609224), (1360192353, 0.21257355388323368, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192320] : ((1360192320, -2.768958503016805, 492609224), (1360192320, 0.17793634005207853, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192290] : ((1360192290, -1.1381338548989424, 492688767), (1360192290, 0.28818164915649025, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com args[1360192256] : ((1360192256, 1.0568974376182032, 492688767), (1360192256, 0.4331190596735327, 2107752395), '0.08107557331381626') We are sending mail with results at report@fotonower.com refus_total : 0.08107557331381626 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23231814 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23231814_21-05-2025_22_54_17.pdf results_Auto_P23231814_21-05-2025_22_54_17.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23231814_21-05-2025_22_54_17.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23231814','results_Auto_P23231814_21-05-2025_22_54_17.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23231814_21-05-2025_22_54_17.pdf','pdf','','0.57','0.08107557331381626') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23231814

https://www.fotonower.com/image?json=false&list_photos_id=1360193010
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360192824
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360192782
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.5098748969272844)
https://www.fotonower.com/image?json=false&list_photos_id=1360192384
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360192353
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360192320
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360192290
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360192256
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.0568974376182032)

Dans ces conditions,le taux de refus est: 8.11%
Veuillez trouver les photos des contaminants.

exemples de contaminants: metal: https://www.fotonower.com/view/23233051?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23233052?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23233053?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23233056?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23233057?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23233058?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23231814_21-05-2025_22_54_17.pdf.

Lien vers velours :https://www.fotonower.com/velours/23233051,23233052,23233053,23233054,23233055,23233056,23233057,23233058,23233059,23233060,23233061?tags=metal,pet_fonce,papier,flou,background,autre,pet_clair,carton,environnement,mal_croppe,pehd.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 21 May 2025 20:54:25 GMT Content-Length: 0 Connection: close X-Message-Id: W6684NZwSouZJyPpn-eSSw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1360193010, 1360192824, 1360192782, 1360192384, 1360192353, 1360192320, 1360192290, 1360192256] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360193010', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192824', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192782', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192384', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192353', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192320', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192290', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192256', None, None, None, None, None, '2929958') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.012815713882446289 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.984925985336304 time spend to save output : 0.013023138046264648 total time spend for step 9 : 7.997949123382568 step10:split_time_score Wed May 21 22:54:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 21052025 23231814 Nombre de photos uploadées : 8 / 23040 (0%) 21052025 23231814 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 21052025 23231814 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 7.3909759521484375e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.000457763671875 elapsed_time : insert_dashboard_record_day_entry 0.02235555648803711 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.08331856787854033 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219688_21-05-2025_18_56_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219688 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219688 AND mptpi.`type`=3594 To do Qualite : 0.12381409868474136 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219689_21-05-2025_19_06_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219689 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219689 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219717 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219758 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219775 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219778 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219783 order by id desc limit 1 Qualite : 0.08107557331381626 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23231814_21-05-2025_22_54_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231814 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23231814 AND mptpi.`type`=3594 To do Qualite : 0.12656074474060458 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219791_21-05-2025_20_04_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219791 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219791 AND mptpi.`type`=3594 To do Qualite : 0.1417862796160131 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219795_21-05-2025_18_37_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219795 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219795 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219802 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231823 order by id desc limit 1 Qualite : 0.13299991003345785 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23231842_21-05-2025_22_45_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231842 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23231842 AND mptpi.`type`=3594 To do Qualite : 0.09422273973246707 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23219879_21-05-2025_18_18_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219879 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23219879 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23219881 order by id desc limit 1 Qualite : 0.1368348169707698 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23223954_21-05-2025_19_55_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23223954 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23223954 AND mptpi.`type`=3594 To do Qualite : 0.11925225240559187 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23223969_21-05-2025_19_45_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23223969 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23223969 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231873 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231874 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231876 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231879 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23231882 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23229467 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'21052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1360193010, 1360192824, 1360192782, 1360192384, 1360192353, 1360192320, 1360192290, 1360192256] Looping around the photos to save general results len do output : 1 /23231814Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360193010', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192824', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192782', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192384', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192353', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192320', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192290', None, None, None, None, None, '2929958') ('3318', None, None, None, None, None, None, None, '2929958') ('3318', '23231814', '1360192256', None, None, None, None, None, '2929958') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.017519235610961914 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.045100927352905 time spend to save output : 0.0177462100982666 total time spend for step 10 : 13.062847137451172 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 92.24user 108.78system 4:13.28elapsed 79%CPU (0avgtext+0avgdata 4661560maxresident)k 150160inputs+58360outputs (10317major+9572934minor)pagefaults 0swaps