python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2807605 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2909104'] with mtr_portfolio_ids : ['23170104'] and first list_photo_ids : [] new path : /proc/2807605/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFB[Errno 104] Connection reset by peer can't download the photo : 1359796976 complete_path : /home/admin/workarea/git/Velours/python/temp/1747747830_2807605_1359796976_db153c60c9ece65b90426fb6b04fe23d.jpg FBFBFBFBFwe have missing 1 photos in the step downloads : photo missing : [1359796976] try to delete the photos missing in DB Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1359796976 length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 1.2164537906646729 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue May 20 15:30:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-20 15:30:34.873909: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-20 15:30:34.882457: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-20 15:30:34.884043: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f72ac000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-20 15:30:34.884105: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-20 15:30:34.886824: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-20 15:30:34.999218: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4becd60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-20 15:30:34.999290: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-20 15:30:35.000844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-20 15:30:35.001323: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 15:30:35.004535: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 15:30:35.007631: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-20 15:30:35.008262: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-20 15:30:35.011807: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-20 15:30:35.012985: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-20 15:30:35.017518: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-20 15:30:35.019108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-20 15:30:35.019220: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 15:30:35.020044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-20 15:30:35.020061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-20 15:30:35.020071: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-20 15:30:35.021765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9671 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-20 15:30:35.401067: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-20 15:30:35.401200: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 15:30:35.401220: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 15:30:35.401238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-20 15:30:35.401256: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-20 15:30:35.401273: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-20 15:30:35.401289: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-20 15:30:35.401307: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-20 15:30:35.402584: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-20 15:30:35.404119: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-20 15:30:35.404191: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 15:30:35.404207: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 15:30:35.404222: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-20 15:30:35.404236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-20 15:30:35.404250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-20 15:30:35.404265: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-20 15:30:35.404279: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-20 15:30:35.405531: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-20 15:30:35.405579: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-20 15:30:35.405588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-20 15:30:35.405596: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-20 15:30:35.406963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9671 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-20 15:30:47.724556: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 15:30:47.918273: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 52 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 43 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 37 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 33 Detection mask done ! Trying to reset tf kernel 2808366 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1527 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6816 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.002475738525390625 nb_pixel_total : 64234 time to create 1 rle with old method : 0.07626605033874512 length of segment : 195 time for calcul the mask position with numpy : 0.005500316619873047 nb_pixel_total : 185142 time to create 1 rle with new method : 0.013422250747680664 length of segment : 709 time for calcul the mask position with numpy : 0.0019779205322265625 nb_pixel_total : 48116 time to create 1 rle with old method : 0.054825782775878906 length of segment : 313 time for calcul the mask position with numpy : 0.0009961128234863281 nb_pixel_total : 41136 time to create 1 rle with old method : 0.046914100646972656 length of segment : 207 time for calcul the mask position with numpy : 0.0005519390106201172 nb_pixel_total : 20778 time to create 1 rle with old method : 0.02386188507080078 length of segment : 262 time for calcul the mask position with numpy : 0.0008463859558105469 nb_pixel_total : 38412 time to create 1 rle with old method : 0.046019792556762695 length of segment : 188 time for calcul the mask position with numpy : 0.0008213520050048828 nb_pixel_total : 33781 time to create 1 rle with old method : 0.04122304916381836 length of segment : 154 time for calcul the mask position with numpy : 0.0005986690521240234 nb_pixel_total : 19701 time to create 1 rle with old method : 0.03251791000366211 length of segment : 243 time for calcul the mask position with numpy : 0.0014498233795166016 nb_pixel_total : 53846 time to create 1 rle with old method : 0.06155896186828613 length of segment : 275 time for calcul the mask position with numpy : 0.0002727508544921875 nb_pixel_total : 7047 time to create 1 rle with old method : 0.008399248123168945 length of segment : 127 time for calcul the mask position with numpy : 0.0011191368103027344 nb_pixel_total : 49782 time to create 1 rle with old method : 0.06010603904724121 length of segment : 416 time for calcul the mask position with numpy : 0.0014145374298095703 nb_pixel_total : 64512 time to create 1 rle with old method : 0.08449077606201172 length of segment : 260 time for calcul the mask position with numpy : 0.00025272369384765625 nb_pixel_total : 8278 time to create 1 rle with old method : 0.009694814682006836 length of segment : 98 time for calcul the mask position with numpy : 0.002438783645629883 nb_pixel_total : 77874 time to create 1 rle with old method : 0.09667778015136719 length of segment : 556 time for calcul the mask position with numpy : 0.0009360313415527344 nb_pixel_total : 33261 time to create 1 rle with old method : 0.0378415584564209 length of segment : 240 time for calcul the mask position with numpy : 0.004784584045410156 nb_pixel_total : 149541 time to create 1 rle with old method : 0.1685802936553955 length of segment : 405 time for calcul the mask position with numpy : 0.0004265308380126953 nb_pixel_total : 14366 time to create 1 rle with old method : 0.01647210121154785 length of segment : 156 time for calcul the mask position with numpy : 0.0008578300476074219 nb_pixel_total : 35454 time to create 1 rle with old method : 0.04091978073120117 length of segment : 288 time for calcul the mask position with numpy : 0.0010302066802978516 nb_pixel_total : 33925 time to create 1 rle with old method : 0.04827594757080078 length of segment : 220 time for calcul the mask position with numpy : 0.0012784004211425781 nb_pixel_total : 19623 time to create 1 rle with old method : 0.02282238006591797 length of segment : 296 time for calcul the mask position with numpy : 0.0010013580322265625 nb_pixel_total : 25817 time to create 1 rle with old method : 0.02971029281616211 length of segment : 299 time for calcul the mask position with numpy : 0.004247903823852539 nb_pixel_total : 168569 time to create 1 rle with new method : 0.015494346618652344 length of segment : 558 time for calcul the mask position with numpy : 0.002060413360595703 nb_pixel_total : 58334 time to create 1 rle with old method : 0.06709909439086914 length of segment : 355 time for calcul the mask position with numpy : 0.0025289058685302734 nb_pixel_total : 65421 time to create 1 rle with old method : 0.08957862854003906 length of segment : 596 time for calcul the mask position with numpy : 0.0015873908996582031 nb_pixel_total : 63999 time to create 1 rle with old method : 0.07844948768615723 length of segment : 283 time for calcul the mask position with numpy : 0.0021810531616210938 nb_pixel_total : 51838 time to create 1 rle with old method : 0.05950188636779785 length of segment : 328 time for calcul the mask position with numpy : 0.0015139579772949219 nb_pixel_total : 60441 time to create 1 rle with old method : 0.068878173828125 length of segment : 320 time for calcul the mask position with numpy : 0.0009312629699707031 nb_pixel_total : 34490 time to create 1 rle with old method : 0.041756391525268555 length of segment : 196 time for calcul the mask position with numpy : 0.0006575584411621094 nb_pixel_total : 19303 time to create 1 rle with old method : 0.02398967742919922 length of segment : 193 time for calcul the mask position with numpy : 0.0012640953063964844 nb_pixel_total : 48481 time to create 1 rle with old method : 0.05548691749572754 length of segment : 228 time for calcul the mask position with numpy : 0.0008082389831542969 nb_pixel_total : 29141 time to create 1 rle with old method : 0.033765554428100586 length of segment : 200 time for calcul the mask position with numpy : 0.0035698413848876953 nb_pixel_total : 64884 time to create 1 rle with old method : 0.07564449310302734 length of segment : 310 time for calcul the mask position with numpy : 0.0013473033905029297 nb_pixel_total : 45993 time to create 1 rle with old method : 0.05266094207763672 length of segment : 274 time for calcul the mask position with numpy : 0.0008902549743652344 nb_pixel_total : 9065 time to create 1 rle with old method : 0.010567903518676758 length of segment : 209 time for calcul the mask position with numpy : 0.001398324966430664 nb_pixel_total : 56107 time to create 1 rle with old method : 0.06442642211914062 length of segment : 289 time for calcul the mask position with numpy : 0.0009210109710693359 nb_pixel_total : 25309 time to create 1 rle with old method : 0.0364835262298584 length of segment : 290 time for calcul the mask position with numpy : 0.0012729167938232422 nb_pixel_total : 42710 time to create 1 rle with old method : 0.04967141151428223 length of segment : 290 time for calcul the mask position with numpy : 0.0017812252044677734 nb_pixel_total : 65600 time to create 1 rle with old method : 0.09541869163513184 length of segment : 312 time for calcul the mask position with numpy : 0.0012099742889404297 nb_pixel_total : 39826 time to create 1 rle with old method : 0.0460667610168457 length of segment : 322 time for calcul the mask position with numpy : 0.0007531642913818359 nb_pixel_total : 27503 time to create 1 rle with old method : 0.03334641456604004 length of segment : 198 time for calcul the mask position with numpy : 0.0016205310821533203 nb_pixel_total : 27348 time to create 1 rle with old method : 0.03194379806518555 length of segment : 376 time for calcul the mask position with numpy : 0.00925588607788086 nb_pixel_total : 224362 time to create 1 rle with new method : 0.028092145919799805 length of segment : 1024 time for calcul the mask position with numpy : 0.0046617984771728516 nb_pixel_total : 181435 time to create 1 rle with new method : 0.01533198356628418 length of segment : 607 time for calcul the mask position with numpy : 0.0007338523864746094 nb_pixel_total : 30724 time to create 1 rle with old method : 0.036885976791381836 length of segment : 218 time for calcul the mask position with numpy : 0.003728151321411133 nb_pixel_total : 111152 time to create 1 rle with old method : 0.13142848014831543 length of segment : 416 time for calcul the mask position with numpy : 0.001142740249633789 nb_pixel_total : 34392 time to create 1 rle with old method : 0.05415177345275879 length of segment : 251 time for calcul the mask position with numpy : 0.0020263195037841797 nb_pixel_total : 60627 time to create 1 rle with old method : 0.07003974914550781 length of segment : 619 time for calcul the mask position with numpy : 0.003067493438720703 nb_pixel_total : 56733 time to create 1 rle with old method : 0.06510734558105469 length of segment : 230 time for calcul the mask position with numpy : 0.0009043216705322266 nb_pixel_total : 10355 time to create 1 rle with old method : 0.016173839569091797 length of segment : 174 time for calcul the mask position with numpy : 0.004015207290649414 nb_pixel_total : 62527 time to create 1 rle with old method : 0.07456421852111816 length of segment : 411 time for calcul the mask position with numpy : 0.001132965087890625 nb_pixel_total : 18734 time to create 1 rle with old method : 0.022819042205810547 length of segment : 156 time for calcul the mask position with numpy : 0.0026166439056396484 nb_pixel_total : 48203 time to create 1 rle with old method : 0.07357978820800781 length of segment : 272 time for calcul the mask position with numpy : 0.002592325210571289 nb_pixel_total : 47989 time to create 1 rle with old method : 0.0569305419921875 length of segment : 323 time for calcul the mask position with numpy : 0.006875514984130859 nb_pixel_total : 123893 time to create 1 rle with old method : 0.14248132705688477 length of segment : 600 time for calcul the mask position with numpy : 0.002588033676147461 nb_pixel_total : 41452 time to create 1 rle with old method : 0.0653538703918457 length of segment : 323 time for calcul the mask position with numpy : 0.006535530090332031 nb_pixel_total : 76863 time to create 1 rle with old method : 0.09213042259216309 length of segment : 409 time for calcul the mask position with numpy : 0.0040624141693115234 nb_pixel_total : 80994 time to create 1 rle with old method : 0.09955215454101562 length of segment : 334 time for calcul the mask position with numpy : 0.003290414810180664 nb_pixel_total : 102280 time to create 1 rle with old method : 0.11935019493103027 length of segment : 378 time for calcul the mask position with numpy : 0.004073619842529297 nb_pixel_total : 48736 time to create 1 rle with old method : 0.12848997116088867 length of segment : 619 time for calcul the mask position with numpy : 0.004626035690307617 nb_pixel_total : 115682 time to create 1 rle with old method : 0.13026118278503418 length of segment : 516 time for calcul the mask position with numpy : 0.0020635128021240234 nb_pixel_total : 56417 time to create 1 rle with old method : 0.06383633613586426 length of segment : 316 time for calcul the mask position with numpy : 0.0025010108947753906 nb_pixel_total : 39231 time to create 1 rle with old method : 0.062378644943237305 length of segment : 430 time for calcul the mask position with numpy : 0.002632617950439453 nb_pixel_total : 60845 time to create 1 rle with old method : 0.06949377059936523 length of segment : 401 time for calcul the mask position with numpy : 0.001580953598022461 nb_pixel_total : 44594 time to create 1 rle with old method : 0.050696372985839844 length of segment : 267 time for calcul the mask position with numpy : 0.0005846023559570312 nb_pixel_total : 11669 time to create 1 rle with old method : 0.01389765739440918 length of segment : 104 time for calcul the mask position with numpy : 0.00252532958984375 nb_pixel_total : 59963 time to create 1 rle with old method : 0.07603883743286133 length of segment : 226 time for calcul the mask position with numpy : 0.003117799758911133 nb_pixel_total : 86069 time to create 1 rle with old method : 0.10559248924255371 length of segment : 320 time for calcul the mask position with numpy : 0.0035216808319091797 nb_pixel_total : 91507 time to create 1 rle with old method : 0.1035757064819336 length of segment : 489 time for calcul the mask position with numpy : 0.004127979278564453 nb_pixel_total : 127531 time to create 1 rle with old method : 0.1447124481201172 length of segment : 484 time for calcul the mask position with numpy : 0.0057735443115234375 nb_pixel_total : 156977 time to create 1 rle with new method : 0.01271200180053711 length of segment : 536 time for calcul the mask position with numpy : 0.0016455650329589844 nb_pixel_total : 35109 time to create 1 rle with old method : 0.04157376289367676 length of segment : 193 time for calcul the mask position with numpy : 0.0011756420135498047 nb_pixel_total : 29357 time to create 1 rle with old method : 0.03761124610900879 length of segment : 191 time for calcul the mask position with numpy : 0.0029282569885253906 nb_pixel_total : 105992 time to create 1 rle with old method : 0.11934185028076172 length of segment : 249 time for calcul the mask position with numpy : 0.0018794536590576172 nb_pixel_total : 70040 time to create 1 rle with old method : 0.07985401153564453 length of segment : 223 time for calcul the mask position with numpy : 0.0021657943725585938 nb_pixel_total : 33317 time to create 1 rle with old method : 0.03844165802001953 length of segment : 493 time for calcul the mask position with numpy : 0.0013971328735351562 nb_pixel_total : 50931 time to create 1 rle with old method : 0.05791020393371582 length of segment : 639 time for calcul the mask position with numpy : 0.000514984130859375 nb_pixel_total : 21789 time to create 1 rle with old method : 0.026900291442871094 length of segment : 231 time for calcul the mask position with numpy : 0.0012669563293457031 nb_pixel_total : 27457 time to create 1 rle with old method : 0.031531333923339844 length of segment : 192 time for calcul the mask position with numpy : 0.0030579566955566406 nb_pixel_total : 84420 time to create 1 rle with old method : 0.09497237205505371 length of segment : 429 time for calcul the mask position with numpy : 0.0011637210845947266 nb_pixel_total : 25187 time to create 1 rle with old method : 0.029144763946533203 length of segment : 182 time for calcul the mask position with numpy : 0.0007262229919433594 nb_pixel_total : 21033 time to create 1 rle with old method : 0.024771928787231445 length of segment : 171 time spent for convertir_results : 9.77694582939148 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 175 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 3473 save missing photos in datou_result : time spend for datou_step_exec : 58.43571424484253 time spend to save output : 0.5905137062072754 total time spend for step 1 : 59.026227951049805 step2:crop_condition Tue May 20 15:31:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 175 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 114 About to insert : list_path_to_insert length 114 new photo from crops ! About to upload 114 photos upload in portfolio : 3736932 init cache_photo without model_param we have 114 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747747919_2807605 we have uploaded 114 photos in the portfolio 3736932 time of upload the photos Elapsed time : 26.366443872451782 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747747948_2807605 we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.4401021003723145 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747747952_2807605 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8973734378814697 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! About to upload 25 photos upload in portfolio : 3736932 init cache_photo without model_param we have 25 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747747965_2807605 we have uploaded 25 photos in the portfolio 3736932 time of upload the photos Elapsed time : 11.831847429275513 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3736932 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747747981_2807605 we have uploaded 12 photos in the portfolio 3736932 time of upload the photos Elapsed time : 17.05011010169983 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3736932 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747748000_2807605 we have uploaded 4 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.1712756156921387 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3736932 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747748005_2807605 we have uploaded 8 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.1952807903289795 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359796979, 1359796974, 1359796971, 1359796945, 1359796943] Looping around the photos to save general results len do output : 170 /1359818593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818605Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818621Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818633Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818659Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818660Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818663Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818670Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818674Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818702Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818712Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818723Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359818813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796979', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796974', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796971', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796945', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796943', None, None, None, None, None, '2909104') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 515 time used for this insertion : 0.032953500747680664 save_final save missing photos in datou_result : time spend for datou_step_exec : 116.17459750175476 time spend to save output : 0.0388951301574707 total time spend for step 2 : 116.21349263191223 step3:rle_unique_nms_with_priority Tue May 20 15:33:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 175 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 9.379001140594482 time for calcul the mask position with numpy : 0.8344075679779053 nb_pixel_total : 7659698 time to create 1 rle with new method : 1.2020790576934814 time for calcul the mask position with numpy : 0.03311753273010254 nb_pixel_total : 8279 time to create 1 rle with old method : 0.009237527847290039 time for calcul the mask position with numpy : 0.03413963317871094 nb_pixel_total : 185068 time to create 1 rle with new method : 0.7627191543579102 time for calcul the mask position with numpy : 0.034369468688964844 nb_pixel_total : 64237 time to create 1 rle with old method : 0.07151365280151367 time for calcul the mask position with numpy : 0.03464198112487793 nb_pixel_total : 48129 time to create 1 rle with old method : 0.054657697677612305 time for calcul the mask position with numpy : 0.035678863525390625 nb_pixel_total : 7045 time to create 1 rle with old method : 0.007998466491699219 time for calcul the mask position with numpy : 0.03443121910095215 nb_pixel_total : 19699 time to create 1 rle with old method : 0.022297143936157227 time for calcul the mask position with numpy : 0.0343780517578125 nb_pixel_total : 38411 time to create 1 rle with old method : 0.042999267578125 time for calcul the mask position with numpy : 0.03443336486816406 nb_pixel_total : 64506 time to create 1 rle with old method : 0.07200741767883301 time for calcul the mask position with numpy : 0.03452634811401367 nb_pixel_total : 53848 time to create 1 rle with old method : 0.06067633628845215 time for calcul the mask position with numpy : 0.036164283752441406 nb_pixel_total : 49783 time to create 1 rle with old method : 0.05746865272521973 time for calcul the mask position with numpy : 0.03638482093811035 nb_pixel_total : 20772 time to create 1 rle with old method : 0.0234220027923584 time for calcul the mask position with numpy : 0.03457832336425781 nb_pixel_total : 41142 time to create 1 rle with old method : 0.04777026176452637 time for calcul the mask position with numpy : 0.034857988357543945 nb_pixel_total : 33783 time to create 1 rle with old method : 0.03908348083496094 create new chi : 3.8612725734710693 time to delete rle : 0.015613079071044922 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.09355616569519043 nb_obj : 25 nb_hashtags : 6 time to prepare the origin masks : 9.28596568107605 time for calcul the mask position with numpy : 0.6249964237213135 nb_pixel_total : 7141645 time to create 1 rle with new method : 0.6341345310211182 time for calcul the mask position with numpy : 0.034770965576171875 nb_pixel_total : 64907 time to create 1 rle with old method : 0.07301998138427734 time for calcul the mask position with numpy : 0.038164615631103516 nb_pixel_total : 168565 time to create 1 rle with new method : 1.3164856433868408 time for calcul the mask position with numpy : 0.03433537483215332 nb_pixel_total : 35453 time to create 1 rle with old method : 0.03964829444885254 time for calcul the mask position with numpy : 0.034296274185180664 nb_pixel_total : 9063 time to create 1 rle with old method : 0.010217666625976562 time for calcul the mask position with numpy : 0.03423285484313965 nb_pixel_total : 51832 time to create 1 rle with old method : 0.05769848823547363 time for calcul the mask position with numpy : 0.03437662124633789 nb_pixel_total : 58309 time to create 1 rle with old method : 0.08472251892089844 time for calcul the mask position with numpy : 0.03774762153625488 nb_pixel_total : 25319 time to create 1 rle with old method : 0.030694007873535156 time for calcul the mask position with numpy : 0.03538966178894043 nb_pixel_total : 65637 time to create 1 rle with old method : 0.0861668586730957 time for calcul the mask position with numpy : 0.0363616943359375 nb_pixel_total : 19308 time to create 1 rle with old method : 0.02168750762939453 time for calcul the mask position with numpy : 0.03407001495361328 nb_pixel_total : 34492 time to create 1 rle with old method : 0.04229092597961426 time for calcul the mask position with numpy : 0.03458142280578613 nb_pixel_total : 45994 time to create 1 rle with old method : 0.0512235164642334 time for calcul the mask position with numpy : 0.03402876853942871 nb_pixel_total : 19625 time to create 1 rle with old method : 0.021992206573486328 time for calcul the mask position with numpy : 0.034235477447509766 nb_pixel_total : 78067 time to create 1 rle with old method : 0.08826541900634766 time for calcul the mask position with numpy : 0.034085750579833984 nb_pixel_total : 29138 time to create 1 rle with old method : 0.03383207321166992 time for calcul the mask position with numpy : 0.04419374465942383 nb_pixel_total : 63994 time to create 1 rle with old method : 0.07690048217773438 time for calcul the mask position with numpy : 0.03437972068786621 nb_pixel_total : 33821 time to create 1 rle with old method : 0.038039207458496094 time for calcul the mask position with numpy : 0.03424215316772461 nb_pixel_total : 193 time to create 1 rle with old method : 0.00044727325439453125 time for calcul the mask position with numpy : 0.03433966636657715 nb_pixel_total : 149525 time to create 1 rle with old method : 0.16805720329284668 time for calcul the mask position with numpy : 0.03470492362976074 nb_pixel_total : 33265 time to create 1 rle with old method : 0.04747438430786133 time for calcul the mask position with numpy : 0.03482794761657715 nb_pixel_total : 56099 time to create 1 rle with old method : 0.062441349029541016 time for calcul the mask position with numpy : 0.03553056716918945 nb_pixel_total : 26098 time to create 1 rle with old method : 0.04221343994140625 time for calcul the mask position with numpy : 0.038738250732421875 nb_pixel_total : 60298 time to create 1 rle with old method : 0.06708216667175293 time for calcul the mask position with numpy : 0.03443598747253418 nb_pixel_total : 9392 time to create 1 rle with old method : 0.01073908805847168 time for calcul the mask position with numpy : 0.03417849540710449 nb_pixel_total : 14361 time to create 1 rle with old method : 0.016112804412841797 create new chi : 4.699208498001099 time to delete rle : 0.002640247344970703 batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++Number RLEs to save : 363 TO DO : save crop sub photo not yet done ! save time : 0.17302632331848145 nb_obj : 18 nb_hashtags : 5 time to prepare the origin masks : 9.381863355636597 time for calcul the mask position with numpy : 1.006913185119629 nb_pixel_total : 7252238 time to create 1 rle with new method : 1.160921335220337 time for calcul the mask position with numpy : 0.0344538688659668 nb_pixel_total : 10352 time to create 1 rle with old method : 0.011676788330078125 time for calcul the mask position with numpy : 0.033707618713378906 nb_pixel_total : 56753 time to create 1 rle with old method : 0.06285667419433594 time for calcul the mask position with numpy : 0.035057783126831055 nb_pixel_total : 181432 time to create 1 rle with new method : 0.9727966785430908 time for calcul the mask position with numpy : 0.03348541259765625 nb_pixel_total : 39826 time to create 1 rle with old method : 0.04431939125061035 time for calcul the mask position with numpy : 0.03430438041687012 nb_pixel_total : 48205 time to create 1 rle with old method : 0.05386161804199219 time for calcul the mask position with numpy : 0.035761117935180664 nb_pixel_total : 62526 time to create 1 rle with old method : 0.07272553443908691 time for calcul the mask position with numpy : 0.03436398506164551 nb_pixel_total : 60610 time to create 1 rle with old method : 0.06650662422180176 time for calcul the mask position with numpy : 0.03519487380981445 nb_pixel_total : 224401 time to create 1 rle with new method : 1.0522270202636719 time for calcul the mask position with numpy : 0.04950523376464844 nb_pixel_total : 18734 time to create 1 rle with old method : 0.020979881286621094 time for calcul the mask position with numpy : 0.03415393829345703 nb_pixel_total : 27346 time to create 1 rle with old method : 0.030681371688842773 time for calcul the mask position with numpy : 0.03395867347717285 nb_pixel_total : 236 time to create 1 rle with old method : 0.0005195140838623047 time for calcul the mask position with numpy : 0.03390645980834961 nb_pixel_total : 42356 time to create 1 rle with old method : 0.048451900482177734 time for calcul the mask position with numpy : 0.03510928153991699 nb_pixel_total : 27503 time to create 1 rle with old method : 0.034459590911865234 time for calcul the mask position with numpy : 0.03519558906555176 nb_pixel_total : 65602 time to create 1 rle with old method : 0.07646656036376953 time for calcul the mask position with numpy : 0.03593730926513672 nb_pixel_total : 30723 time to create 1 rle with old method : 0.03585195541381836 time for calcul the mask position with numpy : 0.040130615234375 nb_pixel_total : 111162 time to create 1 rle with old method : 0.12561297416687012 time for calcul the mask position with numpy : 0.034337759017944336 nb_pixel_total : 34395 time to create 1 rle with old method : 0.0385584831237793 create new chi : 5.6742658615112305 time to delete rle : 0.0021903514862060547 batch 1 Loaded 36 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 422 TO DO : save crop sub photo not yet done ! save time : 0.16040372848510742 nb_obj : 16 nb_hashtags : 6 time to prepare the origin masks : 9.083317041397095 time for calcul the mask position with numpy : 0.764103889465332 nb_pixel_total : 7371736 time to create 1 rle with new method : 0.8042111396789551 time for calcul the mask position with numpy : 0.0333864688873291 nb_pixel_total : 81046 time to create 1 rle with old method : 0.08780884742736816 time for calcul the mask position with numpy : 0.03276348114013672 nb_pixel_total : 44592 time to create 1 rle with old method : 0.04792475700378418 time for calcul the mask position with numpy : 0.03337979316711426 nb_pixel_total : 11670 time to create 1 rle with old method : 0.012781858444213867 time for calcul the mask position with numpy : 0.03375983238220215 nb_pixel_total : 123961 time to create 1 rle with old method : 0.13693857192993164 time for calcul the mask position with numpy : 0.03326916694641113 nb_pixel_total : 38291 time to create 1 rle with old method : 0.04190564155578613 time for calcul the mask position with numpy : 0.033247947692871094 nb_pixel_total : 115680 time to create 1 rle with old method : 0.12527155876159668 time for calcul the mask position with numpy : 0.033588409423828125 nb_pixel_total : 102304 time to create 1 rle with old method : 0.11921024322509766 time for calcul the mask position with numpy : 0.0347900390625 nb_pixel_total : 56418 time to create 1 rle with old method : 0.06628179550170898 time for calcul the mask position with numpy : 0.036804914474487305 nb_pixel_total : 47419 time to create 1 rle with old method : 0.053904056549072266 time for calcul the mask position with numpy : 0.034774065017700195 nb_pixel_total : 747 time to create 1 rle with old method : 0.0009813308715820312 time for calcul the mask position with numpy : 0.0349576473236084 nb_pixel_total : 41555 time to create 1 rle with old method : 0.047144412994384766 time for calcul the mask position with numpy : 0.034328460693359375 nb_pixel_total : 36080 time to create 1 rle with old method : 0.04100346565246582 time for calcul the mask position with numpy : 0.0346829891204834 nb_pixel_total : 60817 time to create 1 rle with old method : 0.07025980949401855 time for calcul the mask position with numpy : 0.03499627113342285 nb_pixel_total : 76007 time to create 1 rle with old method : 0.08880853652954102 time for calcul the mask position with numpy : 0.03651261329650879 nb_pixel_total : 86077 time to create 1 rle with old method : 0.1022801399230957 create new chi : 3.1982290744781494 time to delete rle : 0.0018537044525146484 batch 1 Loaded 38 chid ids of type : 3594 ++++++++++++++++++++++++++++++Number RLEs to save : 461 TO DO : save crop sub photo not yet done ! save time : 0.18707561492919922 nb_obj : 15 nb_hashtags : 5 time to prepare the origin masks : 9.696622848510742 time for calcul the mask position with numpy : 0.8558249473571777 nb_pixel_total : 7417671 time to create 1 rle with new method : 1.0382733345031738 time for calcul the mask position with numpy : 0.03480339050292969 nb_pixel_total : 127500 time to create 1 rle with old method : 0.14306402206420898 time for calcul the mask position with numpy : 0.034447431564331055 nb_pixel_total : 33313 time to create 1 rle with old method : 0.03752422332763672 time for calcul the mask position with numpy : 0.03524184226989746 nb_pixel_total : 106008 time to create 1 rle with old method : 0.1191706657409668 time for calcul the mask position with numpy : 0.034960031509399414 nb_pixel_total : 156975 time to create 1 rle with new method : 1.6051409244537354 time for calcul the mask position with numpy : 0.034522056579589844 nb_pixel_total : 84420 time to create 1 rle with old method : 0.09407877922058105 time for calcul the mask position with numpy : 0.034067392349243164 nb_pixel_total : 29356 time to create 1 rle with old method : 0.03310561180114746 time for calcul the mask position with numpy : 0.03429388999938965 nb_pixel_total : 91501 time to create 1 rle with old method : 0.10212993621826172 time for calcul the mask position with numpy : 0.03414750099182129 nb_pixel_total : 47041 time to create 1 rle with old method : 0.05315232276916504 time for calcul the mask position with numpy : 0.034604549407958984 nb_pixel_total : 21791 time to create 1 rle with old method : 0.02431797981262207 time for calcul the mask position with numpy : 0.0340268611907959 nb_pixel_total : 27457 time to create 1 rle with old method : 0.030659914016723633 time for calcul the mask position with numpy : 0.03438425064086914 nb_pixel_total : 70041 time to create 1 rle with old method : 0.07755303382873535 time for calcul the mask position with numpy : 0.03419184684753418 nb_pixel_total : 35106 time to create 1 rle with old method : 0.03914833068847656 time for calcul the mask position with numpy : 0.03403496742248535 nb_pixel_total : 25189 time to create 1 rle with old method : 0.028109073638916016 time for calcul the mask position with numpy : 0.03407788276672363 nb_pixel_total : 21031 time to create 1 rle with old method : 0.023497343063354492 create new chi : 4.892497777938843 time to delete rle : 0.0021805763244628906 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.10517477989196777 map_output_result : {1359796979: (0.0, 'Should be the crop_list due to order', 0), 1359796974: (0.0, 'Should be the crop_list due to order', 0), 1359796971: (0.0, 'Should be the crop_list due to order', 0), 1359796945: (0.0, 'Should be the crop_list due to order', 0), 1359796943: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359796979, 1359796974, 1359796971, 1359796945, 1359796943] Looping around the photos to save general results len do output : 5 /1359796979.Didn't retrieve data . /1359796974.Didn't retrieve data . /1359796971.Didn't retrieve data . /1359796945.Didn't retrieve data . /1359796943.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796979', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796974', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796971', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796945', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796943', None, None, None, None, None, '2909104') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.013885259628295898 save_final save missing photos in datou_result : time spend for datou_step_exec : 70.85219883918762 time spend to save output : 0.014267206192016602 total time spend for step 3 : 70.86646604537964 step4:ventilate_hashtags_in_portfolio Tue May 20 15:34:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23170104 get user id for portfolio 23170104 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170104 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','flou','mal_croppe','pet_clair','pehd','carton','environnement','pet_fonce','papier','metal','autre')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170104 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','flou','mal_croppe','pet_clair','pehd','carton','environnement','pet_fonce','papier','metal','autre')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170104 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','flou','mal_croppe','pet_clair','pehd','carton','environnement','pet_fonce','papier','metal','autre')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23170980,23170981,23170982,23170983,23170984,23170985,23170986,23170987,23170988,23170989,23170990?tags=background,flou,mal_croppe,pet_clair,pehd,carton,environnement,pet_fonce,papier,metal,autre Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359796979, 1359796974, 1359796971, 1359796945, 1359796943] Looping around the photos to save general results len do output : 1 /23170104. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796979', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796974', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796971', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796945', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796943', None, None, None, None, None, '2909104') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013368606567382812 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5942802429199219 time spend to save output : 0.013643026351928711 total time spend for step 4 : 1.6079232692718506 step5:final Tue May 20 15:34:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359796979: ('0.11795572916666669',), 1359796974: ('0.11795572916666669',), 1359796971: ('0.11795572916666669',), 1359796945: ('0.11795572916666669',), 1359796943: ('0.11795572916666669',)} new output for save of step final : {1359796979: ('0.11795572916666669',), 1359796974: ('0.11795572916666669',), 1359796971: ('0.11795572916666669',), 1359796945: ('0.11795572916666669',), 1359796943: ('0.11795572916666669',)} [1359796979, 1359796974, 1359796971, 1359796945, 1359796943] Looping around the photos to save general results len do output : 5 /1359796979.Didn't retrieve data . /1359796974.Didn't retrieve data . /1359796971.Didn't retrieve data . /1359796945.Didn't retrieve data . /1359796943.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796979', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796974', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796971', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796945', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796943', None, None, None, None, None, '2909104') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.011890411376953125 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11632728576660156 time spend to save output : 0.012235164642333984 total time spend for step 5 : 0.12856245040893555 step6:blur_detection Tue May 20 15:34:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 5 time used for this insertion : 0.009026050567626953 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 5 time used for this insertion : 0.00978398323059082 save missing photos in datou_result : time spend for datou_step_exec : 0.021346330642700195 time spend to save output : 0.023854494094848633 total time spend for step 6 : 0.04520082473754883 step7:brightness Tue May 20 15:34:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 5 time used for this insertion : 0.009072065353393555 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 5 time used for this insertion : 0.009787797927856445 save missing photos in datou_result : time spend for datou_step_exec : 0.025588512420654297 time spend to save output : 0.023303985595703125 total time spend for step 7 : 0.04889249801635742 step8:velours_tree Tue May 20 15:34:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.10212540626525879 time spend to save output : 4.9114227294921875e-05 total time spend for step 8 : 0.10217452049255371 step9:send_mail_cod Tue May 20 15:34:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23170104_20-05-2025_15_34_40.pdf 23170980 imagette231709801747748080 23170981 imagette231709811747748080 23170982 imagette231709821747748080 23170983 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231709831747748080 23170984 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231709841747748081 23170985 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231709851747748082 23170987 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231709871747748082 23170988 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text -- Missing : http://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/20/f6c22e5f2b8cac4c8164415d3f135944_33289019.png retrying : [Errno 104] Connection reset by peer change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231709881747748083 23170989 change filename to text .change filename to text .imagette231709891747748085 23170990 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231709901747748085 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23170104 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23170980,23170981,23170982,23170983,23170984,23170985,23170986,23170987,23170988,23170989,23170990?tags=background,flou,mal_croppe,pet_clair,pehd,carton,environnement,pet_fonce,papier,metal,autre args[1359796979] : ((1359796979, -5.605630903289912, 492609224), (1359796979, 0.480404319613089, 2107752395), '0.11795572916666669') We are sending mail with results at report@fotonower.com args[1359796974] : ((1359796974, -5.693550276172795, 492609224), (1359796974, 0.8641675712045704, 2107752395), '0.11795572916666669') We are sending mail with results at report@fotonower.com args[1359796971] : ((1359796971, -5.7404721984331335, 492609224), (1359796971, 0.8201690026728361, 2107752395), '0.11795572916666669') We are sending mail with results at report@fotonower.com args[1359796945] : ((1359796945, -5.497608111626438, 492609224), (1359796945, 0.7622986881869046, 2107752395), '0.11795572916666669') We are sending mail with results at report@fotonower.com args[1359796943] : ((1359796943, -5.676084325162087, 492609224), (1359796943, 0.5211445417030505, 2107752395), '0.11795572916666669') We are sending mail with results at report@fotonower.com refus_total : 0.11795572916666669 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23170104 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 error cannot identify image file 'imagette231709881747748083/1747747830_2807605_1359796979_a85251babef820d40e19141a8ba49c3a_rle_crop_3806327759_0.png.png' start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170104_20-05-2025_15_34_40.pdf results_Auto_P23170104_20-05-2025_15_34_40.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170104_20-05-2025_15_34_40.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23170104','results_Auto_P23170104_20-05-2025_15_34_40.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170104_20-05-2025_15_34_40.pdf','pdf','','0.62','0.11795572916666669') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23170104

https://www.fotonower.com/image?json=false&list_photos_id=1359796979
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359796974
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359796971
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359796945
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359796943
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 11.80%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/23170983?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/23170984?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23170985?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23170987?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23170988?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23170989?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23170990?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170104_20-05-2025_15_34_40.pdf.

Lien vers velours :https://www.fotonower.com/velours/23170980,23170981,23170982,23170983,23170984,23170985,23170986,23170987,23170988,23170989,23170990?tags=background,flou,mal_croppe,pet_clair,pehd,carton,environnement,pet_fonce,papier,metal,autre.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 20 May 2025 13:34:48 GMT Content-Length: 0 Connection: close X-Message-Id: CMAFLKnJSh-Pj9Zw-Wdt3Q Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359796979, 1359796974, 1359796971, 1359796945, 1359796943] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796979', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796974', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796971', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796945', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796943', None, None, None, None, None, '2909104') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.012467622756958008 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.4109468460083 time spend to save output : 0.012681722640991211 total time spend for step 9 : 8.423628568649292 step10:split_time_score Tue May 20 15:34:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 6),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20052025 23170104 Nombre de photos uploadées : 6 / 23040 (0%) 20052025 23170104 Nombre de photos taguées (types de déchets): 0 / 6 (0%) 20052025 23170104 Nombre de photos taguées (volume) : 0 / 6 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 7.62939453125e-06 ?????? elapsed_time : fill_and_build_computed_from_old_data 0.0004150867462158203 elapsed_time : insert_dashboard_record_day_entry 0.02174973487854004 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.09201400421027917 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23157181_20-05-2025_08_46_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23157181 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23157181 AND mptpi.`type`=3594 To do Qualite : 0.10180426772060878 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23157188_20-05-2025_08_35_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23157188 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23157188 AND mptpi.`type`=3594 To do Qualite : 0.09871742380401231 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23157192_20-05-2025_08_28_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23157192 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23157192 AND mptpi.`type`=3594 To do Qualite : 0.10408518822159352 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23161557_20-05-2025_10_30_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23161557 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23161557 AND mptpi.`type`=3594 To do Qualite : 0.1308608699845679 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23164737_20-05-2025_11_54_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23164737 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23164737 AND mptpi.`type`=3594 To do Qualite : 0.1064864810896508 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23164744_20-05-2025_12_03_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23164744 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23164744 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170102 order by id desc limit 1 Qualite : 0.11795572916666669 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170104_20-05-2025_15_34_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170104 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170104 AND mptpi.`type`=3594 To do Qualite : 0.02726871178050528 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23166958_20-05-2025_13_30_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23166958 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23166958 AND mptpi.`type`=3726 To do Qualite : 0.09543954354745371 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23166961_20-05-2025_13_20_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23166961 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23166961 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170110 order by id desc limit 1 Qualite : 0.10659472484016758 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170112_20-05-2025_15_29_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170112 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170112 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20052025': {'nb_upload': 6, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359796979, 1359796974, 1359796971, 1359796945, 1359796943] Looping around the photos to save general results len do output : 1 /23170104Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796979', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796974', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796971', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796945', None, None, None, None, None, '2909104') ('3318', None, None, None, None, None, None, None, '2909104') ('3318', '23170104', '1359796943', None, None, None, None, None, '2909104') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013094425201416016 save_final save missing photos in datou_result : begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.010748624801635742 time spend for datou_step_exec : 2.4946861267089844 time spend to save output : 0.024033784866333008 total time spend for step 10 : 2.5187199115753174 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 6 set_done_treatment 136.77user 55.89system 4:25.34elapsed 72%CPU (0avgtext+0avgdata 4866764maxresident)k 660232inputs+159216outputs (6631major+10000512minor)pagefaults 0swaps