python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 573924 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2652445'] with mtr_portfolio_ids : ['21179437'] and first list_photo_ids : [] new path : /proc/573924/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 2.049654722213745 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Sat Mar 8 13:50:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9928 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-08 13:50:35.429960: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-08 13:50:35.455068: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-08 13:50:35.457187: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f1de0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-08 13:50:35.457248: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-08 13:50:35.461391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-08 13:50:35.735775: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x12dde110 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-08 13:50:35.735828: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-08 13:50:35.737319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-08 13:50:35.738096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-08 13:50:35.743094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-08 13:50:35.746476: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-08 13:50:35.748083: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-08 13:50:35.751401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-08 13:50:35.753031: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-08 13:50:35.759249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-08 13:50:35.760870: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-08 13:50:35.760971: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-08 13:50:35.761823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-08 13:50:35.761844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-08 13:50:35.761854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-08 13:50:35.763325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9151 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-08 13:50:36.098703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-08 13:50:36.098844: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-08 13:50:36.098878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-08 13:50:36.098951: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-08 13:50:36.098985: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-08 13:50:36.099014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-08 13:50:36.099041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-08 13:50:36.099069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-08 13:50:36.100849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-08 13:50:36.102616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-08 13:50:36.102711: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-08 13:50:36.102738: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-08 13:50:36.102762: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-08 13:50:36.102787: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-08 13:50:36.102811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-08 13:50:36.102835: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-08 13:50:36.102859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-08 13:50:36.105567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-08 13:50:36.105621: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-08 13:50:36.105636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-08 13:50:36.105648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-08 13:50:36.108551: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9151 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-08 13:50:49.012373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-08 13:50:49.216033: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 44 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 57 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 48 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 61 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.35000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 43 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 40 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.60000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 32 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 148.85000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 34 Detection mask done ! Trying to reset tf kernel 574669 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3225 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9707 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.01992011070251465 nb_pixel_total : 32991 time to create 1 rle with old method : 0.046408891677856445 length of segment : 180 time for calcul the mask position with numpy : 0.01957559585571289 nb_pixel_total : 22707 time to create 1 rle with old method : 0.02678394317626953 length of segment : 179 time for calcul the mask position with numpy : 0.006348371505737305 nb_pixel_total : 7646 time to create 1 rle with old method : 0.013657569885253906 length of segment : 82 time for calcul the mask position with numpy : 0.0036385059356689453 nb_pixel_total : 9372 time to create 1 rle with old method : 0.013514995574951172 length of segment : 118 time for calcul the mask position with numpy : 0.01715993881225586 nb_pixel_total : 18484 time to create 1 rle with old method : 0.025812625885009766 length of segment : 179 time for calcul the mask position with numpy : 0.031934261322021484 nb_pixel_total : 42470 time to create 1 rle with old method : 0.052168846130371094 length of segment : 292 time for calcul the mask position with numpy : 0.11242341995239258 nb_pixel_total : 172576 time to create 1 rle with new method : 0.01355123519897461 length of segment : 643 time for calcul the mask position with numpy : 0.009650945663452148 nb_pixel_total : 19707 time to create 1 rle with old method : 0.027654170989990234 length of segment : 151 time for calcul the mask position with numpy : 0.013847827911376953 nb_pixel_total : 20497 time to create 1 rle with old method : 0.030315160751342773 length of segment : 223 time for calcul the mask position with numpy : 0.12599873542785645 nb_pixel_total : 209858 time to create 1 rle with new method : 0.017612218856811523 length of segment : 583 time for calcul the mask position with numpy : 0.04822039604187012 nb_pixel_total : 64530 time to create 1 rle with old method : 0.08603429794311523 length of segment : 393 time for calcul the mask position with numpy : 0.0032701492309570312 nb_pixel_total : 6043 time to create 1 rle with old method : 0.010908365249633789 length of segment : 83 time for calcul the mask position with numpy : 0.013634204864501953 nb_pixel_total : 62938 time to create 1 rle with old method : 0.07815980911254883 length of segment : 526 time for calcul the mask position with numpy : 0.008748769760131836 nb_pixel_total : 11136 time to create 1 rle with old method : 0.01696324348449707 length of segment : 155 time for calcul the mask position with numpy : 0.03543996810913086 nb_pixel_total : 32776 time to create 1 rle with old method : 0.06314587593078613 length of segment : 253 time for calcul the mask position with numpy : 0.028616905212402344 nb_pixel_total : 27895 time to create 1 rle with old method : 0.04151344299316406 length of segment : 259 time for calcul the mask position with numpy : 0.04518914222717285 nb_pixel_total : 140707 time to create 1 rle with old method : 0.16898703575134277 length of segment : 373 time for calcul the mask position with numpy : 0.01310420036315918 nb_pixel_total : 12754 time to create 1 rle with old method : 0.025734901428222656 length of segment : 112 time for calcul the mask position with numpy : 0.002375364303588867 nb_pixel_total : 15918 time to create 1 rle with old method : 0.019671201705932617 length of segment : 130 time for calcul the mask position with numpy : 0.003769397735595703 nb_pixel_total : 54856 time to create 1 rle with old method : 0.08229351043701172 length of segment : 238 time for calcul the mask position with numpy : 0.0038568973541259766 nb_pixel_total : 41268 time to create 1 rle with old method : 0.04958462715148926 length of segment : 260 time for calcul the mask position with numpy : 0.0012731552124023438 nb_pixel_total : 4278 time to create 1 rle with old method : 0.005025625228881836 length of segment : 104 time for calcul the mask position with numpy : 0.0011298656463623047 nb_pixel_total : 12973 time to create 1 rle with old method : 0.015249013900756836 length of segment : 155 time for calcul the mask position with numpy : 0.0006978511810302734 nb_pixel_total : 4232 time to create 1 rle with old method : 0.005032062530517578 length of segment : 93 time for calcul the mask position with numpy : 0.007639408111572266 nb_pixel_total : 23917 time to create 1 rle with old method : 0.028088092803955078 length of segment : 135 time for calcul the mask position with numpy : 0.17874932289123535 nb_pixel_total : 228418 time to create 1 rle with new method : 0.01830744743347168 length of segment : 751 time for calcul the mask position with numpy : 0.03224611282348633 nb_pixel_total : 28493 time to create 1 rle with old method : 0.04137253761291504 length of segment : 250 time for calcul the mask position with numpy : 0.011232614517211914 nb_pixel_total : 150490 time to create 1 rle with new method : 0.01048588752746582 length of segment : 415 time for calcul the mask position with numpy : 0.0043447017669677734 nb_pixel_total : 33916 time to create 1 rle with old method : 0.043694257736206055 length of segment : 336 time for calcul the mask position with numpy : 0.007348775863647461 nb_pixel_total : 56954 time to create 1 rle with old method : 0.07497787475585938 length of segment : 454 time for calcul the mask position with numpy : 0.005338907241821289 nb_pixel_total : 20375 time to create 1 rle with old method : 0.032166481018066406 length of segment : 225 time for calcul the mask position with numpy : 0.006253957748413086 nb_pixel_total : 30529 time to create 1 rle with old method : 0.04075932502746582 length of segment : 223 time for calcul the mask position with numpy : 0.028736114501953125 nb_pixel_total : 28934 time to create 1 rle with old method : 0.03737950325012207 length of segment : 203 time for calcul the mask position with numpy : 0.0004286766052246094 nb_pixel_total : 4773 time to create 1 rle with old method : 0.006197452545166016 length of segment : 79 time for calcul the mask position with numpy : 0.0058324337005615234 nb_pixel_total : 56650 time to create 1 rle with old method : 0.06595611572265625 length of segment : 256 time for calcul the mask position with numpy : 0.12982940673828125 nb_pixel_total : 262087 time to create 1 rle with new method : 0.38663673400878906 length of segment : 919 time for calcul the mask position with numpy : 0.027474641799926758 nb_pixel_total : 47548 time to create 1 rle with old method : 0.05944371223449707 length of segment : 249 time for calcul the mask position with numpy : 0.011573076248168945 nb_pixel_total : 106874 time to create 1 rle with old method : 0.134246826171875 length of segment : 431 time for calcul the mask position with numpy : 0.019207239151000977 nb_pixel_total : 77763 time to create 1 rle with old method : 0.09496641159057617 length of segment : 367 time for calcul the mask position with numpy : 0.003941059112548828 nb_pixel_total : 37410 time to create 1 rle with old method : 0.048218488693237305 length of segment : 255 time for calcul the mask position with numpy : 0.007580280303955078 nb_pixel_total : 63297 time to create 1 rle with old method : 0.13674664497375488 length of segment : 341 time for calcul the mask position with numpy : 0.0049974918365478516 nb_pixel_total : 44085 time to create 1 rle with old method : 0.08294105529785156 length of segment : 397 time for calcul the mask position with numpy : 0.0012662410736083984 nb_pixel_total : 20396 time to create 1 rle with old method : 0.03467512130737305 length of segment : 126 time for calcul the mask position with numpy : 0.004285573959350586 nb_pixel_total : 5741 time to create 1 rle with old method : 0.009831666946411133 length of segment : 138 time for calcul the mask position with numpy : 0.003398418426513672 nb_pixel_total : 36381 time to create 1 rle with old method : 0.06032586097717285 length of segment : 293 time for calcul the mask position with numpy : 0.0027468204498291016 nb_pixel_total : 15843 time to create 1 rle with old method : 0.02642226219177246 length of segment : 207 time for calcul the mask position with numpy : 0.001272439956665039 nb_pixel_total : 8464 time to create 1 rle with old method : 0.014703512191772461 length of segment : 119 time for calcul the mask position with numpy : 0.0018968582153320312 nb_pixel_total : 7939 time to create 1 rle with old method : 0.013689994812011719 length of segment : 171 time for calcul the mask position with numpy : 0.001485586166381836 nb_pixel_total : 14923 time to create 1 rle with old method : 0.023461341857910156 length of segment : 176 time for calcul the mask position with numpy : 0.006274223327636719 nb_pixel_total : 66423 time to create 1 rle with old method : 0.12810826301574707 length of segment : 318 time for calcul the mask position with numpy : 0.0024232864379882812 nb_pixel_total : 16964 time to create 1 rle with old method : 0.04390549659729004 length of segment : 182 time for calcul the mask position with numpy : 0.0029401779174804688 nb_pixel_total : 15564 time to create 1 rle with old method : 0.030883312225341797 length of segment : 232 time for calcul the mask position with numpy : 0.0009405612945556641 nb_pixel_total : 9404 time to create 1 rle with old method : 0.01865696907043457 length of segment : 139 time for calcul the mask position with numpy : 0.010226249694824219 nb_pixel_total : 85674 time to create 1 rle with old method : 0.16777396202087402 length of segment : 338 time for calcul the mask position with numpy : 0.0009751319885253906 nb_pixel_total : 16515 time to create 1 rle with old method : 0.03255152702331543 length of segment : 266 time for calcul the mask position with numpy : 0.0012302398681640625 nb_pixel_total : 15033 time to create 1 rle with old method : 0.029607772827148438 length of segment : 167 time for calcul the mask position with numpy : 0.0005567073822021484 nb_pixel_total : 10913 time to create 1 rle with old method : 0.020020484924316406 length of segment : 165 time for calcul the mask position with numpy : 0.014513492584228516 nb_pixel_total : 193420 time to create 1 rle with new method : 0.021956682205200195 length of segment : 643 time for calcul the mask position with numpy : 0.0022950172424316406 nb_pixel_total : 25086 time to create 1 rle with old method : 0.04647946357727051 length of segment : 204 time for calcul the mask position with numpy : 0.00047087669372558594 nb_pixel_total : 7179 time to create 1 rle with old method : 0.010961055755615234 length of segment : 95 time for calcul the mask position with numpy : 0.0013973712921142578 nb_pixel_total : 23817 time to create 1 rle with old method : 0.03975033760070801 length of segment : 245 time for calcul the mask position with numpy : 0.0019519329071044922 nb_pixel_total : 29333 time to create 1 rle with old method : 0.03588986396789551 length of segment : 258 time for calcul the mask position with numpy : 0.0011224746704101562 nb_pixel_total : 9922 time to create 1 rle with old method : 0.01720571517944336 length of segment : 189 time for calcul the mask position with numpy : 0.0008416175842285156 nb_pixel_total : 10207 time to create 1 rle with old method : 0.020011186599731445 length of segment : 132 time for calcul the mask position with numpy : 0.001317739486694336 nb_pixel_total : 16479 time to create 1 rle with old method : 0.029591083526611328 length of segment : 153 time for calcul the mask position with numpy : 0.0013840198516845703 nb_pixel_total : 27978 time to create 1 rle with old method : 0.034449100494384766 length of segment : 183 time for calcul the mask position with numpy : 0.0012845993041992188 nb_pixel_total : 23289 time to create 1 rle with old method : 0.02968454360961914 length of segment : 226 time for calcul the mask position with numpy : 0.0031180381774902344 nb_pixel_total : 61310 time to create 1 rle with old method : 0.07687807083129883 length of segment : 389 time for calcul the mask position with numpy : 0.0018305778503417969 nb_pixel_total : 34041 time to create 1 rle with old method : 0.04190540313720703 length of segment : 274 time for calcul the mask position with numpy : 0.001386404037475586 nb_pixel_total : 22253 time to create 1 rle with old method : 0.02675318717956543 length of segment : 200 time for calcul the mask position with numpy : 0.00400996208190918 nb_pixel_total : 65351 time to create 1 rle with old method : 0.08110260963439941 length of segment : 335 time for calcul the mask position with numpy : 0.002610921859741211 nb_pixel_total : 62711 time to create 1 rle with old method : 0.0917353630065918 length of segment : 280 time for calcul the mask position with numpy : 0.0005960464477539062 nb_pixel_total : 15022 time to create 1 rle with old method : 0.019003868103027344 length of segment : 125 time for calcul the mask position with numpy : 0.0004487037658691406 nb_pixel_total : 5912 time to create 1 rle with old method : 0.007700920104980469 length of segment : 86 time for calcul the mask position with numpy : 0.0003540515899658203 nb_pixel_total : 6623 time to create 1 rle with old method : 0.008464813232421875 length of segment : 113 time for calcul the mask position with numpy : 0.00489497184753418 nb_pixel_total : 81503 time to create 1 rle with old method : 0.09688973426818848 length of segment : 358 time for calcul the mask position with numpy : 0.002915620803833008 nb_pixel_total : 33781 time to create 1 rle with old method : 0.04084014892578125 length of segment : 444 time for calcul the mask position with numpy : 0.001439809799194336 nb_pixel_total : 29140 time to create 1 rle with old method : 0.0374908447265625 length of segment : 205 time for calcul the mask position with numpy : 0.0007789134979248047 nb_pixel_total : 12244 time to create 1 rle with old method : 0.015372276306152344 length of segment : 104 time for calcul the mask position with numpy : 0.0012309551239013672 nb_pixel_total : 29483 time to create 1 rle with old method : 0.03533577919006348 length of segment : 137 time for calcul the mask position with numpy : 0.0008051395416259766 nb_pixel_total : 7737 time to create 1 rle with old method : 0.010147809982299805 length of segment : 140 time for calcul the mask position with numpy : 0.0006084442138671875 nb_pixel_total : 8448 time to create 1 rle with old method : 0.010590076446533203 length of segment : 101 time for calcul the mask position with numpy : 0.0005295276641845703 nb_pixel_total : 9031 time to create 1 rle with old method : 0.011333703994750977 length of segment : 174 time for calcul the mask position with numpy : 0.0008995532989501953 nb_pixel_total : 12506 time to create 1 rle with old method : 0.015011787414550781 length of segment : 168 time for calcul the mask position with numpy : 0.0005784034729003906 nb_pixel_total : 8014 time to create 1 rle with old method : 0.010032892227172852 length of segment : 106 time for calcul the mask position with numpy : 0.0036280155181884766 nb_pixel_total : 63892 time to create 1 rle with old method : 0.07411074638366699 length of segment : 712 time for calcul the mask position with numpy : 0.0031414031982421875 nb_pixel_total : 44872 time to create 1 rle with old method : 0.05533766746520996 length of segment : 333 time for calcul the mask position with numpy : 0.001468658447265625 nb_pixel_total : 22760 time to create 1 rle with old method : 0.034774065017700195 length of segment : 385 time for calcul the mask position with numpy : 0.0012683868408203125 nb_pixel_total : 20531 time to create 1 rle with old method : 0.026230812072753906 length of segment : 202 time for calcul the mask position with numpy : 0.0018601417541503906 nb_pixel_total : 23676 time to create 1 rle with old method : 0.029918432235717773 length of segment : 355 time for calcul the mask position with numpy : 0.002753734588623047 nb_pixel_total : 26415 time to create 1 rle with old method : 0.03510022163391113 length of segment : 177 time for calcul the mask position with numpy : 0.00035953521728515625 nb_pixel_total : 4606 time to create 1 rle with old method : 0.00569605827331543 length of segment : 129 time for calcul the mask position with numpy : 0.00042057037353515625 nb_pixel_total : 6921 time to create 1 rle with old method : 0.009167671203613281 length of segment : 107 time for calcul the mask position with numpy : 0.0005695819854736328 nb_pixel_total : 10902 time to create 1 rle with old method : 0.014124631881713867 length of segment : 154 time for calcul the mask position with numpy : 0.0013074874877929688 nb_pixel_total : 39731 time to create 1 rle with old method : 0.04896426200866699 length of segment : 221 time for calcul the mask position with numpy : 0.0035796165466308594 nb_pixel_total : 84466 time to create 1 rle with old method : 0.105499267578125 length of segment : 367 time for calcul the mask position with numpy : 0.0015895366668701172 nb_pixel_total : 36113 time to create 1 rle with old method : 0.04462289810180664 length of segment : 402 time for calcul the mask position with numpy : 0.0007319450378417969 nb_pixel_total : 10860 time to create 1 rle with old method : 0.012624979019165039 length of segment : 133 time for calcul the mask position with numpy : 0.0010025501251220703 nb_pixel_total : 24515 time to create 1 rle with old method : 0.030462026596069336 length of segment : 171 time for calcul the mask position with numpy : 0.001283407211303711 nb_pixel_total : 21337 time to create 1 rle with old method : 0.02485489845275879 length of segment : 186 time for calcul the mask position with numpy : 0.002551555633544922 nb_pixel_total : 75386 time to create 1 rle with old method : 0.09356975555419922 length of segment : 396 time for calcul the mask position with numpy : 0.0016388893127441406 nb_pixel_total : 42748 time to create 1 rle with old method : 0.05139565467834473 length of segment : 197 time for calcul the mask position with numpy : 0.000789642333984375 nb_pixel_total : 17564 time to create 1 rle with old method : 0.021581172943115234 length of segment : 204 time for calcul the mask position with numpy : 0.0009539127349853516 nb_pixel_total : 31738 time to create 1 rle with old method : 0.04097437858581543 length of segment : 241 time for calcul the mask position with numpy : 0.0007617473602294922 nb_pixel_total : 9564 time to create 1 rle with old method : 0.011890172958374023 length of segment : 132 time for calcul the mask position with numpy : 0.002353191375732422 nb_pixel_total : 60070 time to create 1 rle with old method : 0.07187318801879883 length of segment : 374 time for calcul the mask position with numpy : 0.0057392120361328125 nb_pixel_total : 152187 time to create 1 rle with new method : 0.009886026382446289 length of segment : 545 time for calcul the mask position with numpy : 0.005323648452758789 nb_pixel_total : 158144 time to create 1 rle with new method : 0.006917238235473633 length of segment : 547 time for calcul the mask position with numpy : 0.0007135868072509766 nb_pixel_total : 16666 time to create 1 rle with old method : 0.02042245864868164 length of segment : 141 time for calcul the mask position with numpy : 0.0015172958374023438 nb_pixel_total : 18687 time to create 1 rle with old method : 0.02390122413635254 length of segment : 203 time for calcul the mask position with numpy : 0.0061473846435546875 nb_pixel_total : 110991 time to create 1 rle with old method : 0.13261914253234863 length of segment : 471 time for calcul the mask position with numpy : 0.0023589134216308594 nb_pixel_total : 33290 time to create 1 rle with old method : 0.03864598274230957 length of segment : 347 time for calcul the mask position with numpy : 0.0005280971527099609 nb_pixel_total : 3673 time to create 1 rle with old method : 0.004792451858520508 length of segment : 260 time for calcul the mask position with numpy : 0.0016982555389404297 nb_pixel_total : 38018 time to create 1 rle with old method : 0.04495382308959961 length of segment : 303 time for calcul the mask position with numpy : 0.0008797645568847656 nb_pixel_total : 21025 time to create 1 rle with old method : 0.02719593048095703 length of segment : 186 time for calcul the mask position with numpy : 0.0006651878356933594 nb_pixel_total : 5423 time to create 1 rle with old method : 0.006728649139404297 length of segment : 169 time for calcul the mask position with numpy : 0.00147247314453125 nb_pixel_total : 19719 time to create 1 rle with old method : 0.027901887893676758 length of segment : 310 time for calcul the mask position with numpy : 0.0007736682891845703 nb_pixel_total : 14528 time to create 1 rle with old method : 0.02372908592224121 length of segment : 121 time for calcul the mask position with numpy : 0.0013973712921142578 nb_pixel_total : 18612 time to create 1 rle with old method : 0.025883197784423828 length of segment : 166 time for calcul the mask position with numpy : 0.004469156265258789 nb_pixel_total : 56649 time to create 1 rle with old method : 0.07800841331481934 length of segment : 379 time for calcul the mask position with numpy : 0.00034427642822265625 nb_pixel_total : 4337 time to create 1 rle with old method : 0.0071353912353515625 length of segment : 82 time for calcul the mask position with numpy : 0.001085042953491211 nb_pixel_total : 23388 time to create 1 rle with old method : 0.028607606887817383 length of segment : 162 time for calcul the mask position with numpy : 0.0013301372528076172 nb_pixel_total : 28025 time to create 1 rle with old method : 0.03613710403442383 length of segment : 261 time for calcul the mask position with numpy : 0.0007534027099609375 nb_pixel_total : 9893 time to create 1 rle with old method : 0.012884855270385742 length of segment : 111 time for calcul the mask position with numpy : 0.0019025802612304688 nb_pixel_total : 41203 time to create 1 rle with old method : 0.0510861873626709 length of segment : 308 time for calcul the mask position with numpy : 0.00439763069152832 nb_pixel_total : 91917 time to create 1 rle with old method : 0.13058042526245117 length of segment : 372 time for calcul the mask position with numpy : 0.0004405975341796875 nb_pixel_total : 7767 time to create 1 rle with old method : 0.00974416732788086 length of segment : 103 time for calcul the mask position with numpy : 0.0016388893127441406 nb_pixel_total : 40462 time to create 1 rle with old method : 0.04784870147705078 length of segment : 209 time for calcul the mask position with numpy : 0.0003571510314941406 nb_pixel_total : 3711 time to create 1 rle with old method : 0.004667997360229492 length of segment : 77 time for calcul the mask position with numpy : 0.0008308887481689453 nb_pixel_total : 14440 time to create 1 rle with old method : 0.017869234085083008 length of segment : 132 time for calcul the mask position with numpy : 0.0003809928894042969 nb_pixel_total : 5769 time to create 1 rle with old method : 0.007183551788330078 length of segment : 111 time for calcul the mask position with numpy : 0.0004432201385498047 nb_pixel_total : 7624 time to create 1 rle with old method : 0.009407281875610352 length of segment : 106 time for calcul the mask position with numpy : 0.0007939338684082031 nb_pixel_total : 21884 time to create 1 rle with old method : 0.026102066040039062 length of segment : 217 time for calcul the mask position with numpy : 0.0007083415985107422 nb_pixel_total : 23039 time to create 1 rle with old method : 0.028439998626708984 length of segment : 231 time for calcul the mask position with numpy : 0.0014467239379882812 nb_pixel_total : 37950 time to create 1 rle with old method : 0.0442960262298584 length of segment : 198 time for calcul the mask position with numpy : 0.0007526874542236328 nb_pixel_total : 9665 time to create 1 rle with old method : 0.011899232864379883 length of segment : 162 time for calcul the mask position with numpy : 0.0007960796356201172 nb_pixel_total : 17964 time to create 1 rle with old method : 0.023406028747558594 length of segment : 106 time for calcul the mask position with numpy : 0.004935741424560547 nb_pixel_total : 106604 time to create 1 rle with old method : 0.14842987060546875 length of segment : 551 time for calcul the mask position with numpy : 0.0013217926025390625 nb_pixel_total : 29775 time to create 1 rle with old method : 0.05175638198852539 length of segment : 236 time for calcul the mask position with numpy : 0.00220489501953125 nb_pixel_total : 51125 time to create 1 rle with old method : 0.06335759162902832 length of segment : 198 time for calcul the mask position with numpy : 0.0010020732879638672 nb_pixel_total : 21254 time to create 1 rle with old method : 0.03415203094482422 length of segment : 185 time for calcul the mask position with numpy : 0.0017695426940917969 nb_pixel_total : 18700 time to create 1 rle with old method : 0.03486371040344238 length of segment : 231 time for calcul the mask position with numpy : 0.0009722709655761719 nb_pixel_total : 18912 time to create 1 rle with old method : 0.026569604873657227 length of segment : 134 time for calcul the mask position with numpy : 0.0011463165283203125 nb_pixel_total : 18949 time to create 1 rle with old method : 0.023906707763671875 length of segment : 287 time for calcul the mask position with numpy : 0.0012786388397216797 nb_pixel_total : 19129 time to create 1 rle with old method : 0.03728771209716797 length of segment : 202 time for calcul the mask position with numpy : 0.0009419918060302734 nb_pixel_total : 15586 time to create 1 rle with old method : 0.029584646224975586 length of segment : 137 time for calcul the mask position with numpy : 0.0019006729125976562 nb_pixel_total : 27038 time to create 1 rle with old method : 0.05167365074157715 length of segment : 260 time for calcul the mask position with numpy : 0.0014538764953613281 nb_pixel_total : 22249 time to create 1 rle with old method : 0.03008294105529785 length of segment : 229 time for calcul the mask position with numpy : 0.0012524127960205078 nb_pixel_total : 27113 time to create 1 rle with old method : 0.03261089324951172 length of segment : 189 time for calcul the mask position with numpy : 0.0020487308502197266 nb_pixel_total : 43437 time to create 1 rle with old method : 0.05023193359375 length of segment : 267 time for calcul the mask position with numpy : 0.0013928413391113281 nb_pixel_total : 23166 time to create 1 rle with old method : 0.02825331687927246 length of segment : 187 time for calcul the mask position with numpy : 0.0014963150024414062 nb_pixel_total : 32351 time to create 1 rle with old method : 0.03914189338684082 length of segment : 207 time for calcul the mask position with numpy : 0.0005104541778564453 nb_pixel_total : 8178 time to create 1 rle with old method : 0.010343074798583984 length of segment : 114 time for calcul the mask position with numpy : 0.002748727798461914 nb_pixel_total : 62434 time to create 1 rle with old method : 0.07376742362976074 length of segment : 290 time for calcul the mask position with numpy : 0.0005757808685302734 nb_pixel_total : 12494 time to create 1 rle with old method : 0.017053604125976562 length of segment : 150 time for calcul the mask position with numpy : 0.0027375221252441406 nb_pixel_total : 49151 time to create 1 rle with old method : 0.07445406913757324 length of segment : 333 time for calcul the mask position with numpy : 0.0010225772857666016 nb_pixel_total : 22949 time to create 1 rle with old method : 0.02765941619873047 length of segment : 153 time for calcul the mask position with numpy : 0.0005173683166503906 nb_pixel_total : 14181 time to create 1 rle with old method : 0.023343563079833984 length of segment : 157 time for calcul the mask position with numpy : 0.0010867118835449219 nb_pixel_total : 21704 time to create 1 rle with old method : 0.034647226333618164 length of segment : 155 time for calcul the mask position with numpy : 0.0028333663940429688 nb_pixel_total : 47699 time to create 1 rle with old method : 0.05574941635131836 length of segment : 460 time for calcul the mask position with numpy : 0.0020999908447265625 nb_pixel_total : 31909 time to create 1 rle with old method : 0.04221034049987793 length of segment : 200 time for calcul the mask position with numpy : 0.0004947185516357422 nb_pixel_total : 10047 time to create 1 rle with old method : 0.012276172637939453 length of segment : 140 time for calcul the mask position with numpy : 0.0008370876312255859 nb_pixel_total : 16460 time to create 1 rle with old method : 0.020130157470703125 length of segment : 149 time for calcul the mask position with numpy : 0.0030689239501953125 nb_pixel_total : 67981 time to create 1 rle with old method : 0.08788728713989258 length of segment : 626 time for calcul the mask position with numpy : 0.0006167888641357422 nb_pixel_total : 29207 time to create 1 rle with old method : 0.034062862396240234 length of segment : 227 time for calcul the mask position with numpy : 0.0007276535034179688 nb_pixel_total : 15181 time to create 1 rle with old method : 0.018495559692382812 length of segment : 226 time for calcul the mask position with numpy : 0.0012848377227783203 nb_pixel_total : 34828 time to create 1 rle with old method : 0.045896291732788086 length of segment : 224 time for calcul the mask position with numpy : 0.0005192756652832031 nb_pixel_total : 6617 time to create 1 rle with old method : 0.008659601211547852 length of segment : 74 time for calcul the mask position with numpy : 0.0013785362243652344 nb_pixel_total : 25418 time to create 1 rle with old method : 0.03114032745361328 length of segment : 198 time spent for convertir_results : 15.407761096954346 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 169 chid ids of type : 3594 Number RLEs to save : 41204 save missing photos in datou_result : time spend for datou_step_exec : 123.97951793670654 time spend to save output : 52.298009157180786 total time spend for step 1 : 176.27752709388733 step2:crop_condition Sat Mar 8 13:53:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 169 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 118 About to insert : list_path_to_insert length 118 new photo from crops ! About to upload 118 photos upload in portfolio : 3736932 init cache_photo without model_param we have 118 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438433_573924 we have uploaded 118 photos in the portfolio 3736932 time of upload the photos Elapsed time : 39.83461093902588 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 3736932 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438482_573924 we have uploaded 32 photos in the portfolio 3736932 time of upload the photos Elapsed time : 16.942362785339355 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438501_573924 we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.781362533569336 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3736932 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438507_573924 we have uploaded 7 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.237809896469116 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438511_573924 we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9101808071136475 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438514_573924 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.780322790145874 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741438516_573924 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.502288579940796 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333401235, 1333401228, 1333401222, 1333401216, 1333401212, 1333401209, 1333401102, 1333401091] Looping around the photos to save general results len do output : 169 /1341871738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871739Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871740Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871802Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871908Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1341871964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401235', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401228', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401222', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401216', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401212', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401209', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401102', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401091', None, None, None, None, None, '2652445') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 515 time used for this insertion : 0.17489266395568848 save_final save missing photos in datou_result : time spend for datou_step_exec : 108.89022660255432 time spend to save output : 0.25217103958129883 total time spend for step 2 : 109.14239764213562 step3:rle_unique_nms_with_priority Sat Mar 8 13:55:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 169 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 15.398474931716919 time for calcul the mask position with numpy : 0.837975263595581 nb_pixel_total : 7379313 time to create 1 rle with new method : 1.5977821350097656 time for calcul the mask position with numpy : 0.04056668281555176 nb_pixel_total : 12754 time to create 1 rle with old method : 0.014827728271484375 time for calcul the mask position with numpy : 0.042029380798339844 nb_pixel_total : 140707 time to create 1 rle with old method : 0.1632533073425293 time for calcul the mask position with numpy : 0.044091224670410156 nb_pixel_total : 27895 time to create 1 rle with old method : 0.0326991081237793 time for calcul the mask position with numpy : 0.040094614028930664 nb_pixel_total : 32776 time to create 1 rle with old method : 0.036527395248413086 time for calcul the mask position with numpy : 0.03972744941711426 nb_pixel_total : 11136 time to create 1 rle with old method : 0.013249874114990234 time for calcul the mask position with numpy : 0.04111433029174805 nb_pixel_total : 62938 time to create 1 rle with old method : 0.07028341293334961 time for calcul the mask position with numpy : 0.0404658317565918 nb_pixel_total : 6043 time to create 1 rle with old method : 0.007184505462646484 time for calcul the mask position with numpy : 0.03413844108581543 nb_pixel_total : 64530 time to create 1 rle with old method : 0.07458066940307617 time for calcul the mask position with numpy : 0.0356748104095459 nb_pixel_total : 209858 time to create 1 rle with new method : 1.8482613563537598 time for calcul the mask position with numpy : 0.04062390327453613 nb_pixel_total : 20497 time to create 1 rle with old method : 0.02313685417175293 time for calcul the mask position with numpy : 0.04043173789978027 nb_pixel_total : 19707 time to create 1 rle with old method : 0.026632070541381836 time for calcul the mask position with numpy : 0.040550947189331055 nb_pixel_total : 172576 time to create 1 rle with new method : 0.6679728031158447 time for calcul the mask position with numpy : 0.04225635528564453 nb_pixel_total : 42470 time to create 1 rle with old method : 0.048329830169677734 time for calcul the mask position with numpy : 0.039788246154785156 nb_pixel_total : 18484 time to create 1 rle with old method : 0.02118706703186035 time for calcul the mask position with numpy : 0.041875362396240234 nb_pixel_total : 9372 time to create 1 rle with old method : 0.011054754257202148 time for calcul the mask position with numpy : 0.03989291191101074 nb_pixel_total : 7646 time to create 1 rle with old method : 0.008637428283691406 time for calcul the mask position with numpy : 0.042684078216552734 nb_pixel_total : 22707 time to create 1 rle with old method : 0.026738643646240234 time for calcul the mask position with numpy : 0.03818154335021973 nb_pixel_total : 32991 time to create 1 rle with old method : 0.03754234313964844 create new chi : 6.389119386672974 time to delete rle : 0.023122549057006836 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++Number RLEs to save : 11728 TO DO : save crop sub photo not yet done ! save time : 8.786935329437256 nb_obj : 21 nb_hashtags : 3 time to prepare the origin masks : 16.659196376800537 time for calcul the mask position with numpy : 0.4918947219848633 nb_pixel_total : 7013551 time to create 1 rle with new method : 1.0144224166870117 time for calcul the mask position with numpy : 0.045055389404296875 nb_pixel_total : 77763 time to create 1 rle with old method : 0.09205317497253418 time for calcul the mask position with numpy : 0.04449963569641113 nb_pixel_total : 106874 time to create 1 rle with old method : 0.12509560585021973 time for calcul the mask position with numpy : 0.04456782341003418 nb_pixel_total : 47548 time to create 1 rle with old method : 0.05928945541381836 time for calcul the mask position with numpy : 0.05241107940673828 nb_pixel_total : 262087 time to create 1 rle with new method : 0.7197470664978027 time for calcul the mask position with numpy : 0.048191070556640625 nb_pixel_total : 56650 time to create 1 rle with old method : 0.07351827621459961 time for calcul the mask position with numpy : 0.047637224197387695 nb_pixel_total : 4773 time to create 1 rle with old method : 0.00631260871887207 time for calcul the mask position with numpy : 0.04384350776672363 nb_pixel_total : 28934 time to create 1 rle with old method : 0.03589367866516113 time for calcul the mask position with numpy : 0.04335594177246094 nb_pixel_total : 30529 time to create 1 rle with old method : 0.03976249694824219 time for calcul the mask position with numpy : 0.043291330337524414 nb_pixel_total : 9978 time to create 1 rle with old method : 0.01203775405883789 time for calcul the mask position with numpy : 0.04488801956176758 nb_pixel_total : 56954 time to create 1 rle with old method : 0.0720663070678711 time for calcul the mask position with numpy : 0.04404330253601074 nb_pixel_total : 33916 time to create 1 rle with old method : 0.0386960506439209 time for calcul the mask position with numpy : 0.05428719520568848 nb_pixel_total : 150490 time to create 1 rle with new method : 0.5473048686981201 time for calcul the mask position with numpy : 0.049637556076049805 nb_pixel_total : 28493 time to create 1 rle with old method : 0.03418374061584473 time for calcul the mask position with numpy : 0.04726219177246094 nb_pixel_total : 228418 time to create 1 rle with new method : 0.618110179901123 time for calcul the mask position with numpy : 0.04976511001586914 nb_pixel_total : 23917 time to create 1 rle with old method : 0.04020380973815918 time for calcul the mask position with numpy : 0.04784131050109863 nb_pixel_total : 4232 time to create 1 rle with old method : 0.006154060363769531 time for calcul the mask position with numpy : 0.047348737716674805 nb_pixel_total : 12973 time to create 1 rle with old method : 0.01861715316772461 time for calcul the mask position with numpy : 0.04795575141906738 nb_pixel_total : 4278 time to create 1 rle with old method : 0.06839632987976074 time for calcul the mask position with numpy : 0.04960274696350098 nb_pixel_total : 41268 time to create 1 rle with old method : 0.04887843132019043 time for calcul the mask position with numpy : 0.04901552200317383 nb_pixel_total : 54856 time to create 1 rle with old method : 0.06917786598205566 time for calcul the mask position with numpy : 0.04711508750915527 nb_pixel_total : 15918 time to create 1 rle with old method : 0.018442630767822266 create new chi : 5.3929102420806885 time to delete rle : 0.00403904914855957 batch 1 Loaded 43 chid ids of type : 3594 ++++++++++++++++++++++++++++Number RLEs to save : 14519 TO DO : save crop sub photo not yet done ! save time : 12.271289587020874 nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 9.916664600372314 time for calcul the mask position with numpy : 0.5924437046051025 nb_pixel_total : 7584925 time to create 1 rle with new method : 1.1362292766571045 time for calcul the mask position with numpy : 0.055197954177856445 nb_pixel_total : 25086 time to create 1 rle with old method : 0.03299379348754883 time for calcul the mask position with numpy : 0.05242204666137695 nb_pixel_total : 193420 time to create 1 rle with new method : 1.4663379192352295 time for calcul the mask position with numpy : 0.051021575927734375 nb_pixel_total : 10913 time to create 1 rle with old method : 0.013205528259277344 time for calcul the mask position with numpy : 0.0843963623046875 nb_pixel_total : 15033 time to create 1 rle with old method : 0.019765853881835938 time for calcul the mask position with numpy : 0.055696725845336914 nb_pixel_total : 16515 time to create 1 rle with old method : 0.023224592208862305 time for calcul the mask position with numpy : 0.05748319625854492 nb_pixel_total : 85674 time to create 1 rle with old method : 0.12690949440002441 time for calcul the mask position with numpy : 0.05293536186218262 nb_pixel_total : 9404 time to create 1 rle with old method : 0.011331796646118164 time for calcul the mask position with numpy : 0.04707026481628418 nb_pixel_total : 15564 time to create 1 rle with old method : 0.01912832260131836 time for calcul the mask position with numpy : 0.0446925163269043 nb_pixel_total : 16964 time to create 1 rle with old method : 0.021962881088256836 time for calcul the mask position with numpy : 0.05318641662597656 nb_pixel_total : 66423 time to create 1 rle with old method : 0.08388876914978027 time for calcul the mask position with numpy : 0.057015419006347656 nb_pixel_total : 14923 time to create 1 rle with old method : 0.02174687385559082 time for calcul the mask position with numpy : 0.06603002548217773 nb_pixel_total : 7939 time to create 1 rle with old method : 0.009559392929077148 time for calcul the mask position with numpy : 0.05776619911193848 nb_pixel_total : 8464 time to create 1 rle with old method : 0.01021122932434082 time for calcul the mask position with numpy : 0.05607438087463379 nb_pixel_total : 15843 time to create 1 rle with old method : 0.02187204360961914 time for calcul the mask position with numpy : 0.05724024772644043 nb_pixel_total : 36381 time to create 1 rle with old method : 0.04472064971923828 time for calcul the mask position with numpy : 0.05626177787780762 nb_pixel_total : 5741 time to create 1 rle with old method : 0.009621620178222656 time for calcul the mask position with numpy : 0.05823183059692383 nb_pixel_total : 20396 time to create 1 rle with old method : 0.03399491310119629 time for calcul the mask position with numpy : 0.0670480728149414 nb_pixel_total : 44085 time to create 1 rle with old method : 0.054926156997680664 time for calcul the mask position with numpy : 0.0977165699005127 nb_pixel_total : 63297 time to create 1 rle with old method : 0.17862439155578613 time for calcul the mask position with numpy : 0.05922436714172363 nb_pixel_total : 37410 time to create 1 rle with old method : 0.0486757755279541 create new chi : 5.255889415740967 time to delete rle : 0.004302501678466797 batch 1 Loaded 41 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 11914 TO DO : save crop sub photo not yet done ! save time : 16.221044301986694 nb_obj : 32 nb_hashtags : 3 time to prepare the origin masks : 4.801612854003906 time for calcul the mask position with numpy : 1.6938514709472656 nb_pixel_total : 7451888 time to create 1 rle with new method : 0.5410535335540771 time for calcul the mask position with numpy : 0.03513026237487793 nb_pixel_total : 29483 time to create 1 rle with old method : 0.03651690483093262 time for calcul the mask position with numpy : 0.035361289978027344 nb_pixel_total : 8448 time to create 1 rle with old method : 0.010228633880615234 time for calcul the mask position with numpy : 0.03642416000366211 nb_pixel_total : 8014 time to create 1 rle with old method : 0.0094146728515625 time for calcul the mask position with numpy : 0.03544163703918457 nb_pixel_total : 12506 time to create 1 rle with old method : 0.015100240707397461 time for calcul the mask position with numpy : 0.04049420356750488 nb_pixel_total : 12244 time to create 1 rle with old method : 0.01472330093383789 time for calcul the mask position with numpy : 0.03897738456726074 nb_pixel_total : 29333 time to create 1 rle with old method : 0.042695045471191406 time for calcul the mask position with numpy : 0.03738903999328613 nb_pixel_total : 16479 time to create 1 rle with old method : 0.018982887268066406 time for calcul the mask position with numpy : 0.035126686096191406 nb_pixel_total : 23676 time to create 1 rle with old method : 0.02802753448486328 time for calcul the mask position with numpy : 0.03890180587768555 nb_pixel_total : 61310 time to create 1 rle with old method : 0.07119297981262207 time for calcul the mask position with numpy : 0.03618597984313965 nb_pixel_total : 34041 time to create 1 rle with old method : 0.05502152442932129 time for calcul the mask position with numpy : 0.0393826961517334 nb_pixel_total : 29140 time to create 1 rle with old method : 0.041471242904663086 time for calcul the mask position with numpy : 0.039894819259643555 nb_pixel_total : 23289 time to create 1 rle with old method : 0.03198099136352539 time for calcul the mask position with numpy : 0.03597283363342285 nb_pixel_total : 20531 time to create 1 rle with old method : 0.02642345428466797 time for calcul the mask position with numpy : 0.03786754608154297 nb_pixel_total : 65351 time to create 1 rle with old method : 0.07813596725463867 time for calcul the mask position with numpy : 0.03584861755371094 nb_pixel_total : 26415 time to create 1 rle with old method : 0.03221273422241211 time for calcul the mask position with numpy : 0.03670001029968262 nb_pixel_total : 22253 time to create 1 rle with old method : 0.027109861373901367 time for calcul the mask position with numpy : 0.0370028018951416 nb_pixel_total : 81503 time to create 1 rle with old method : 0.09300923347473145 time for calcul the mask position with numpy : 0.034754037857055664 nb_pixel_total : 22760 time to create 1 rle with old method : 0.027221202850341797 time for calcul the mask position with numpy : 0.034772396087646484 nb_pixel_total : 5912 time to create 1 rle with old method : 0.0071010589599609375 time for calcul the mask position with numpy : 0.035784244537353516 nb_pixel_total : 4788 time to create 1 rle with old method : 0.005892753601074219 time for calcul the mask position with numpy : 0.0351715087890625 nb_pixel_total : 6623 time to create 1 rle with old method : 0.009317159652709961 time for calcul the mask position with numpy : 0.03636598587036133 nb_pixel_total : 44872 time to create 1 rle with old method : 0.051287174224853516 time for calcul the mask position with numpy : 0.038640499114990234 nb_pixel_total : 62711 time to create 1 rle with old method : 0.07274127006530762 time for calcul the mask position with numpy : 0.03485512733459473 nb_pixel_total : 7179 time to create 1 rle with old method : 0.008739233016967773 time for calcul the mask position with numpy : 0.0350184440612793 nb_pixel_total : 22896 time to create 1 rle with old method : 0.03126835823059082 time for calcul the mask position with numpy : 0.03593635559082031 nb_pixel_total : 59274 time to create 1 rle with old method : 0.07223105430603027 time for calcul the mask position with numpy : 0.03598380088806152 nb_pixel_total : 9922 time to create 1 rle with old method : 0.011950016021728516 time for calcul the mask position with numpy : 0.03556227684020996 nb_pixel_total : 30615 time to create 1 rle with old method : 0.03685736656188965 time for calcul the mask position with numpy : 0.03597116470336914 nb_pixel_total : 27978 time to create 1 rle with old method : 0.04284191131591797 time for calcul the mask position with numpy : 0.039849281311035156 nb_pixel_total : 7737 time to create 1 rle with old method : 0.009150266647338867 time for calcul the mask position with numpy : 0.03479957580566406 nb_pixel_total : 10207 time to create 1 rle with old method : 0.011727094650268555 time for calcul the mask position with numpy : 0.03478050231933594 nb_pixel_total : 15022 time to create 1 rle with old method : 0.017871856689453125 create new chi : 4.501000642776489 time to delete rle : 0.004862070083618164 batch 1 Loaded 65 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 16564 TO DO : save crop sub photo not yet done ! save time : 19.347346544265747 nb_obj : 28 nb_hashtags : 4 time to prepare the origin masks : 4.923323154449463 time for calcul the mask position with numpy : 0.911933183670044 nb_pixel_total : 7207585 time to create 1 rle with new method : 0.9993054866790771 time for calcul the mask position with numpy : 0.03384542465209961 nb_pixel_total : 60070 time to create 1 rle with old method : 0.06533336639404297 time for calcul the mask position with numpy : 0.03362441062927246 nb_pixel_total : 18612 time to create 1 rle with old method : 0.02241802215576172 time for calcul the mask position with numpy : 0.0334930419921875 nb_pixel_total : 9564 time to create 1 rle with old method : 0.011291980743408203 time for calcul the mask position with numpy : 0.03580832481384277 nb_pixel_total : 10860 time to create 1 rle with old method : 0.012956619262695312 time for calcul the mask position with numpy : 0.03521084785461426 nb_pixel_total : 17564 time to create 1 rle with old method : 0.021061420440673828 time for calcul the mask position with numpy : 0.03614401817321777 nb_pixel_total : 18687 time to create 1 rle with old method : 0.02423858642578125 time for calcul the mask position with numpy : 0.03428363800048828 nb_pixel_total : 21337 time to create 1 rle with old method : 0.024992704391479492 time for calcul the mask position with numpy : 0.03418684005737305 nb_pixel_total : 19719 time to create 1 rle with old method : 0.02317333221435547 time for calcul the mask position with numpy : 0.03585696220397949 nb_pixel_total : 158144 time to create 1 rle with new method : 1.1839289665222168 time for calcul the mask position with numpy : 0.03505277633666992 nb_pixel_total : 38018 time to create 1 rle with old method : 0.048554182052612305 time for calcul the mask position with numpy : 0.035520315170288086 nb_pixel_total : 84466 time to create 1 rle with old method : 0.10022163391113281 time for calcul the mask position with numpy : 0.03624272346496582 nb_pixel_total : 75386 time to create 1 rle with old method : 0.08583545684814453 time for calcul the mask position with numpy : 0.03696751594543457 nb_pixel_total : 33290 time to create 1 rle with old method : 0.03851509094238281 time for calcul the mask position with numpy : 0.03481483459472656 nb_pixel_total : 110991 time to create 1 rle with old method : 0.14377999305725098 time for calcul the mask position with numpy : 0.0392458438873291 nb_pixel_total : 42748 time to create 1 rle with old method : 0.06941914558410645 time for calcul the mask position with numpy : 0.03726458549499512 nb_pixel_total : 152187 time to create 1 rle with new method : 1.7131445407867432 time for calcul the mask position with numpy : 0.033518075942993164 nb_pixel_total : 21025 time to create 1 rle with old method : 0.02404308319091797 time for calcul the mask position with numpy : 0.03217005729675293 nb_pixel_total : 6921 time to create 1 rle with old method : 0.007669210433959961 time for calcul the mask position with numpy : 0.03243589401245117 nb_pixel_total : 31069 time to create 1 rle with old method : 0.03548026084899902 time for calcul the mask position with numpy : 0.033014535903930664 nb_pixel_total : 16666 time to create 1 rle with old method : 0.01838397979736328 time for calcul the mask position with numpy : 0.032401323318481445 nb_pixel_total : 36113 time to create 1 rle with old method : 0.03992962837219238 time for calcul the mask position with numpy : 0.03428506851196289 nb_pixel_total : 4606 time to create 1 rle with old method : 0.005369424819946289 time for calcul the mask position with numpy : 0.03337287902832031 nb_pixel_total : 10902 time to create 1 rle with old method : 0.011768341064453125 time for calcul the mask position with numpy : 0.03318023681640625 nb_pixel_total : 24515 time to create 1 rle with old method : 0.027444839477539062 time for calcul the mask position with numpy : 0.03325843811035156 nb_pixel_total : 3673 time to create 1 rle with old method : 0.0041353702545166016 time for calcul the mask position with numpy : 0.03269243240356445 nb_pixel_total : 5423 time to create 1 rle with old method : 0.006266117095947266 time for calcul the mask position with numpy : 0.03291606903076172 nb_pixel_total : 39731 time to create 1 rle with old method : 0.04776597023010254 time for calcul the mask position with numpy : 0.03376412391662598 nb_pixel_total : 14528 time to create 1 rle with old method : 0.017132997512817383 create new chi : 6.819369554519653 time to delete rle : 0.004561424255371094 batch 1 Loaded 57 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++Number RLEs to save : 16502 TO DO : save crop sub photo not yet done ! save time : 32.68195581436157 nb_obj : 19 nb_hashtags : 3 time to prepare the origin masks : 6.886929512023926 time for calcul the mask position with numpy : 0.5070436000823975 nb_pixel_total : 7743187 time to create 1 rle with new method : 0.7042884826660156 time for calcul the mask position with numpy : 0.02455735206604004 nb_pixel_total : 106604 time to create 1 rle with old method : 0.12437963485717773 time for calcul the mask position with numpy : 0.024714231491088867 nb_pixel_total : 17964 time to create 1 rle with old method : 0.020137310028076172 time for calcul the mask position with numpy : 0.02443242073059082 nb_pixel_total : 9109 time to create 1 rle with old method : 0.010389566421508789 time for calcul the mask position with numpy : 0.02537703514099121 nb_pixel_total : 37950 time to create 1 rle with old method : 0.04542899131774902 time for calcul the mask position with numpy : 0.03315544128417969 nb_pixel_total : 23039 time to create 1 rle with old method : 0.0272367000579834 time for calcul the mask position with numpy : 0.04883885383605957 nb_pixel_total : 21884 time to create 1 rle with old method : 0.025484085083007812 time for calcul the mask position with numpy : 0.04580974578857422 nb_pixel_total : 7624 time to create 1 rle with old method : 0.00943303108215332 time for calcul the mask position with numpy : 0.03582000732421875 nb_pixel_total : 5541 time to create 1 rle with old method : 0.006795644760131836 time for calcul the mask position with numpy : 0.0417940616607666 nb_pixel_total : 14440 time to create 1 rle with old method : 0.017066001892089844 time for calcul the mask position with numpy : 0.04398488998413086 nb_pixel_total : 3711 time to create 1 rle with old method : 0.004762172698974609 time for calcul the mask position with numpy : 0.03979134559631348 nb_pixel_total : 40462 time to create 1 rle with old method : 0.05094265937805176 time for calcul the mask position with numpy : 0.03923344612121582 nb_pixel_total : 7767 time to create 1 rle with old method : 0.009358406066894531 time for calcul the mask position with numpy : 0.035404205322265625 nb_pixel_total : 91917 time to create 1 rle with old method : 0.10503411293029785 time for calcul the mask position with numpy : 0.03632998466491699 nb_pixel_total : 41203 time to create 1 rle with old method : 0.04619336128234863 time for calcul the mask position with numpy : 0.0331273078918457 nb_pixel_total : 9893 time to create 1 rle with old method : 0.011361837387084961 time for calcul the mask position with numpy : 0.023703336715698242 nb_pixel_total : 27731 time to create 1 rle with old method : 0.031116962432861328 time for calcul the mask position with numpy : 0.024643421173095703 nb_pixel_total : 23388 time to create 1 rle with old method : 0.030533552169799805 time for calcul the mask position with numpy : 0.026175737380981445 nb_pixel_total : 4337 time to create 1 rle with old method : 0.005540132522583008 time for calcul the mask position with numpy : 0.02648329734802246 nb_pixel_total : 56649 time to create 1 rle with old method : 0.07098722457885742 create new chi : 2.5440421104431152 time to delete rle : 0.0015976428985595703 batch 1 Loaded 39 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 9804 TO DO : save crop sub photo not yet done ! save time : 13.105267763137817 nb_obj : 13 nb_hashtags : 4 time to prepare the origin masks : 7.857907056808472 time for calcul the mask position with numpy : 1.6394684314727783 nb_pixel_total : 7957967 time to create 1 rle with new method : 1.3670170307159424 time for calcul the mask position with numpy : 0.054083824157714844 nb_pixel_total : 23166 time to create 1 rle with old method : 0.042745351791381836 time for calcul the mask position with numpy : 0.06214475631713867 nb_pixel_total : 43437 time to create 1 rle with old method : 0.07946443557739258 time for calcul the mask position with numpy : 0.0583648681640625 nb_pixel_total : 27113 time to create 1 rle with old method : 0.0435335636138916 time for calcul the mask position with numpy : 0.04585003852844238 nb_pixel_total : 22249 time to create 1 rle with old method : 0.025434017181396484 time for calcul the mask position with numpy : 0.04637742042541504 nb_pixel_total : 27038 time to create 1 rle with old method : 0.03200221061706543 time for calcul the mask position with numpy : 0.04281878471374512 nb_pixel_total : 15586 time to create 1 rle with old method : 0.017893552780151367 time for calcul the mask position with numpy : 0.050200700759887695 nb_pixel_total : 19129 time to create 1 rle with old method : 0.028416872024536133 time for calcul the mask position with numpy : 0.04454350471496582 nb_pixel_total : 18949 time to create 1 rle with old method : 0.022652387619018555 time for calcul the mask position with numpy : 0.04565572738647461 nb_pixel_total : 18912 time to create 1 rle with old method : 0.021442174911499023 time for calcul the mask position with numpy : 0.0484156608581543 nb_pixel_total : 18700 time to create 1 rle with old method : 0.030904293060302734 time for calcul the mask position with numpy : 0.04694962501525879 nb_pixel_total : 21254 time to create 1 rle with old method : 0.025346994400024414 time for calcul the mask position with numpy : 0.04366469383239746 nb_pixel_total : 51125 time to create 1 rle with old method : 0.05947422981262207 time for calcul the mask position with numpy : 0.049562931060791016 nb_pixel_total : 29775 time to create 1 rle with old method : 0.03557109832763672 create new chi : 4.198408842086792 time to delete rle : 0.0052525997161865234 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 7644 TO DO : save crop sub photo not yet done ! save time : 17.923322916030884 nb_obj : 18 nb_hashtags : 4 time to prepare the origin masks : 9.834234952926636 time for calcul the mask position with numpy : 0.5860593318939209 nb_pixel_total : 7787708 time to create 1 rle with new method : 0.8693163394927979 time for calcul the mask position with numpy : 0.04436635971069336 nb_pixel_total : 25418 time to create 1 rle with old method : 0.030176639556884766 time for calcul the mask position with numpy : 0.03969097137451172 nb_pixel_total : 6617 time to create 1 rle with old method : 0.00875234603881836 time for calcul the mask position with numpy : 0.04161262512207031 nb_pixel_total : 34828 time to create 1 rle with old method : 0.04472184181213379 time for calcul the mask position with numpy : 0.04375886917114258 nb_pixel_total : 15181 time to create 1 rle with old method : 0.02057480812072754 time for calcul the mask position with numpy : 0.04860091209411621 nb_pixel_total : 27984 time to create 1 rle with old method : 0.033307790756225586 time for calcul the mask position with numpy : 0.030790328979492188 nb_pixel_total : 67981 time to create 1 rle with old method : 0.09566521644592285 time for calcul the mask position with numpy : 0.026064395904541016 nb_pixel_total : 16460 time to create 1 rle with old method : 0.0186614990234375 time for calcul the mask position with numpy : 0.027548551559448242 nb_pixel_total : 9623 time to create 1 rle with old method : 0.011425018310546875 time for calcul the mask position with numpy : 0.027601957321166992 nb_pixel_total : 31909 time to create 1 rle with old method : 0.036206722259521484 time for calcul the mask position with numpy : 0.026503801345825195 nb_pixel_total : 47699 time to create 1 rle with old method : 0.061292409896850586 time for calcul the mask position with numpy : 0.026094913482666016 nb_pixel_total : 21704 time to create 1 rle with old method : 0.025565624237060547 time for calcul the mask position with numpy : 0.026412248611450195 nb_pixel_total : 13731 time to create 1 rle with old method : 0.015560388565063477 time for calcul the mask position with numpy : 0.02502298355102539 nb_pixel_total : 22949 time to create 1 rle with old method : 0.025968074798583984 time for calcul the mask position with numpy : 0.02445816993713379 nb_pixel_total : 49151 time to create 1 rle with old method : 0.05604195594787598 time for calcul the mask position with numpy : 0.02477240562438965 nb_pixel_total : 12494 time to create 1 rle with old method : 0.014896869659423828 time for calcul the mask position with numpy : 0.026295900344848633 nb_pixel_total : 62434 time to create 1 rle with old method : 0.07348847389221191 time for calcul the mask position with numpy : 0.0250704288482666 nb_pixel_total : 8178 time to create 1 rle with old method : 0.009700536727905273 time for calcul the mask position with numpy : 0.031149625778198242 nb_pixel_total : 32351 time to create 1 rle with old method : 0.0365452766418457 create new chi : 2.690296173095703 time to delete rle : 0.002032756805419922 batch 1 Loaded 37 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 10166 TO DO : save crop sub photo not yet done ! save time : 24.838315963745117 map_output_result : {1333401235: (0.0, 'Should be the crop_list due to order', 0), 1333401228: (0.0, 'Should be the crop_list due to order', 0), 1333401222: (0.0, 'Should be the crop_list due to order', 0), 1333401216: (0.0, 'Should be the crop_list due to order', 0), 1333401212: (0.0, 'Should be the crop_list due to order', 0), 1333401209: (0.0, 'Should be the crop_list due to order', 0), 1333401102: (0.0, 'Should be the crop_list due to order', 0), 1333401091: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1333401235, 1333401228, 1333401222, 1333401216, 1333401212, 1333401209, 1333401102, 1333401091] Looping around the photos to save general results len do output : 8 /1333401235.Didn't retrieve data . /1333401228.Didn't retrieve data . /1333401222.Didn't retrieve data . /1333401216.Didn't retrieve data . /1333401212.Didn't retrieve data . /1333401209.Didn't retrieve data . /1333401102.Didn't retrieve data . /1333401091.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401235', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401228', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401222', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401216', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401212', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401209', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401102', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401091', None, None, None, None, None, '2652445') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.01373910903930664 save_final save missing photos in datou_result : time spend for datou_step_exec : 263.20259976387024 time spend to save output : 0.2292332649230957 total time spend for step 3 : 263.43183302879333 step4:ventilate_hashtags_in_portfolio Sat Mar 8 13:59:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 21179437 get user id for portfolio 21179437 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179437 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_clair','flou','mal_croppe','papier','background','autre','metal','pet_fonce','environnement','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179437 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_clair','flou','mal_croppe','papier','background','autre','metal','pet_fonce','environnement','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179437 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_clair','flou','mal_croppe','papier','background','autre','metal','pet_fonce','environnement','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21181401,21181402,21181403,21181404,21181405,21181406,21181407,21181408,21181409,21181410,21181411?tags=pehd,pet_clair,flou,mal_croppe,papier,background,autre,metal,pet_fonce,environnement,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1333401235, 1333401228, 1333401222, 1333401216, 1333401212, 1333401209, 1333401102, 1333401091] Looping around the photos to save general results len do output : 1 /21179437. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401235', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401228', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401222', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401216', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401212', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401209', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401102', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401091', None, None, None, None, None, '2652445') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.016367197036743164 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.262721300125122 time spend to save output : 0.017414569854736328 total time spend for step 4 : 2.2801358699798584 step5:final Sat Mar 8 13:59:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1333401235: ('0.09387472270447535',), 1333401228: ('0.09387472270447535',), 1333401222: ('0.09387472270447535',), 1333401216: ('0.09387472270447535',), 1333401212: ('0.09387472270447535',), 1333401209: ('0.09387472270447535',), 1333401102: ('0.09387472270447535',), 1333401091: ('0.09387472270447535',)} new output for save of step final : {1333401235: ('0.09387472270447535',), 1333401228: ('0.09387472270447535',), 1333401222: ('0.09387472270447535',), 1333401216: ('0.09387472270447535',), 1333401212: ('0.09387472270447535',), 1333401209: ('0.09387472270447535',), 1333401102: ('0.09387472270447535',), 1333401091: ('0.09387472270447535',)} [1333401235, 1333401228, 1333401222, 1333401216, 1333401212, 1333401209, 1333401102, 1333401091] Looping around the photos to save general results len do output : 8 /1333401235.Didn't retrieve data . /1333401228.Didn't retrieve data . /1333401222.Didn't retrieve data . /1333401216.Didn't retrieve data . /1333401212.Didn't retrieve data . /1333401209.Didn't retrieve data . /1333401102.Didn't retrieve data . /1333401091.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401235', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401228', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401222', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401216', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401212', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401209', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401102', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401091', None, None, None, None, None, '2652445') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.014286994934082031 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1376805305480957 time spend to save output : 0.015403985977172852 total time spend for step 5 : 0.15308451652526855 step6:blur_detection Sat Mar 8 13:59:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 8 time used for this insertion : 0.009670257568359375 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 8 time used for this insertion : 0.011441946029663086 save missing photos in datou_result : time spend for datou_step_exec : 0.13156366348266602 time spend to save output : 0.026229381561279297 total time spend for step 6 : 0.1577930450439453 step7:brightness Sat Mar 8 13:59:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 8 time used for this insertion : 0.008905172348022461 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 8 time used for this insertion : 0.009552955627441406 save missing photos in datou_result : time spend for datou_step_exec : 0.08122849464416504 time spend to save output : 0.02332615852355957 total time spend for step 7 : 0.10455465316772461 step8:velours_tree Sat Mar 8 13:59:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.6080045700073242 time spend to save output : 3.314018249511719e-05 total time spend for step 8 : 0.6080377101898193 step9:send_mail_cod Sat Mar 8 13:59:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21179437_08-03-2025_13_59_44.pdf 21181401 change filename to text .change filename to text .imagette211814011741438784 21181402 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211814021741438784 21181403 imagette211814031741438785 21181404 imagette211814041741438785 21181405 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211814051741438785 21181406 imagette211814061741438786 21181407 change filename to text .change filename to text .change filename to text .imagette211814071741438786 21181408 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211814081741438787 21181409 change filename to text .imagette211814091741438787 21181411 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette211814111741438787 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21179437 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21181401,21181402,21181403,21181404,21181405,21181406,21181407,21181408,21181409,21181410,21181411?tags=pehd,pet_clair,flou,mal_croppe,papier,background,autre,metal,pet_fonce,environnement,carton args[1333401235] : ((1333401235, -6.11513173761706, 492609224), (1333401235, -0.0975597452892964, 496442774), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401228] : ((1333401228, -6.047876302502338, 492609224), (1333401228, 0.020294095952357486, 2107752395), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401222] : ((1333401222, -6.080057534268917, 492609224), (1333401222, -0.009400949811592741, 2107752395), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401216] : ((1333401216, -6.182434980852241, 492609224), (1333401216, 0.0332377324623931, 2107752395), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401212] : ((1333401212, -6.111547048867758, 492609224), (1333401212, -0.03485428667627134, 2107752395), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401209] : ((1333401209, -6.225593592966737, 492609224), (1333401209, -0.1268125776370622, 496442774), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401102] : ((1333401102, -5.408370190648071, 492609224), (1333401102, -0.5091476418597448, 501862349), '0.09387472270447535') We are sending mail with results at report@fotonower.com args[1333401091] : ((1333401091, -5.2986095234194455, 492609224), (1333401091, -0.5347442557200736, 501862349), '0.09387472270447535') We are sending mail with results at report@fotonower.com refus_total : 0.09387472270447535 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21179437 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179437_08-03-2025_13_59_44.pdf results_Auto_P21179437_08-03-2025_13_59_44.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179437_08-03-2025_13_59_44.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','21179437','results_Auto_P21179437_08-03-2025_13_59_44.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179437_08-03-2025_13_59_44.pdf','pdf','','0.64','0.09387472270447535') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/21179437

https://www.fotonower.com/image?json=false&list_photos_id=1333401235
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401228
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401222
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401216
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401212
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401209
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401102
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1333401091
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 9.39%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pehd: https://www.fotonower.com/view/21181401?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/21181402?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/21181405?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/21181407?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/21181408?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/21181409?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/21181411?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179437_08-03-2025_13_59_44.pdf.

Lien vers velours :https://www.fotonower.com/velours/21181401,21181402,21181403,21181404,21181405,21181406,21181407,21181408,21181409,21181410,21181411?tags=pehd,pet_clair,flou,mal_croppe,papier,background,autre,metal,pet_fonce,environnement,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Sat, 08 Mar 2025 12:59:52 GMT Content-Length: 0 Connection: close X-Message-Id: oVgw0DyFQbenUf97cvgUXg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1333401235, 1333401228, 1333401222, 1333401216, 1333401212, 1333401209, 1333401102, 1333401091] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401235', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401228', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401222', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401216', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401212', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401209', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401102', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401091', None, None, None, None, None, '2652445') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 1.2968614101409912 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.685946226119995 time spend to save output : 1.297116994857788 total time spend for step 9 : 8.983063220977783 step10:split_time_score Sat Mar 8 13:59:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 31012025 21179437 Nombre de photos uploadées : 8 / 23040 (0%) 31012025 21179437 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 31012025 21179437 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 4.76837158203125e-06 elapsed_time : order_list_meta_photo_and_scores 1.0013580322265625e-05 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0006175041198730469 elapsed_time : insert_dashboard_record_day_entry 0.025781631469726562 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.06072010421864514 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179330_08-03-2025_13_20_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179330 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179330 AND mptpi.`type`=3726 To do Qualite : 0.020993640569969257 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179331_08-03-2025_13_27_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179331 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179331 AND mptpi.`type`=3726 To do Qualite : 0.03313981497811913 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179373_08-03-2025_13_27_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179373 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179373 AND mptpi.`type`=3726 To do Qualite : 0.04310599458933935 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179393_08-03-2025_13_04_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179393 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179393 AND mptpi.`type`=3726 To do Qualite : 0.04478070632069951 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179396_08-03-2025_13_13_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179396 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179396 AND mptpi.`type`=3726 To do Qualite : 0.0432735385824555 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179428_08-03-2025_13_05_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179428 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179428 AND mptpi.`type`=3726 To do Qualite : 0.09387472270447535 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179437_08-03-2025_13_59_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179437 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179437 AND mptpi.`type`=3594 To do Qualite : 0.025781178002708682 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21179441_08-03-2025_13_06_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179441 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21179441 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21179448 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'31012025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1333401235, 1333401228, 1333401222, 1333401216, 1333401212, 1333401209, 1333401102, 1333401091] Looping around the photos to save general results len do output : 1 /21179437Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401235', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401228', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401222', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401216', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401212', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401209', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401102', None, None, None, None, None, '2652445') ('3318', None, None, None, None, None, None, None, '2652445') ('3318', '21179437', '1333401091', None, None, None, None, None, '2652445') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.015433788299560547 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.522367000579834 time spend to save output : 0.015660524368286133 total time spend for step 10 : 1.5380275249481201 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 125.71user 163.45system 9:29.90elapsed 50%CPU (0avgtext+0avgdata 4487644maxresident)k 2338928inputs+113024outputs (77978major+18347286minor)pagefaults 0swaps