python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2825132 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2896580'] with mtr_portfolio_ids : ['23127827'] and first list_photo_ids : [] new path : /proc/2825132/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 7 ; length of list_pids : 7 ; length of list_args : 7 time to download the photos : 1.522296667098999 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon May 19 17:50:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-19 17:50:33.901757: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-19 17:50:33.927224: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-19 17:50:33.928708: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2910000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-19 17:50:33.928757: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-19 17:50:33.931788: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-19 17:50:34.077194: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2024bb10 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-19 17:50:34.077237: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-19 17:50:34.078051: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 17:50:34.078433: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 17:50:34.080635: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 17:50:34.082871: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 17:50:34.083250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 17:50:34.085719: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 17:50:34.086929: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 17:50:34.091801: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 17:50:34.093361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 17:50:34.093455: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 17:50:34.094252: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-19 17:50:34.094269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-19 17:50:34.094279: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-19 17:50:34.095693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9463 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-19 17:50:34.371978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 17:50:34.372074: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 17:50:34.372095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 17:50:34.372114: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 17:50:34.372131: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 17:50:34.372148: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 17:50:34.372166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 17:50:34.372183: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 17:50:34.373740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 17:50:34.375116: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-19 17:50:34.375174: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-19 17:50:34.375196: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 17:50:34.375216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-19 17:50:34.375241: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-19 17:50:34.375275: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-19 17:50:34.375308: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-19 17:50:34.375345: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-19 17:50:34.376986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-19 17:50:34.377022: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-19 17:50:34.377033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-19 17:50:34.377043: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-19 17:50:34.378672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9463 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-19 17:50:46.893642: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-19 17:50:47.084326: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 7 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 43 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 43 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 32 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 9.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 44 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 50 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 6.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 18 Detection mask done ! Trying to reset tf kernel 2825936 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5304 tf kernel not reseted sub process len(results) : 7 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 7 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10593 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0006985664367675781 nb_pixel_total : 14117 time to create 1 rle with old method : 0.018663883209228516 length of segment : 162 time for calcul the mask position with numpy : 0.0017518997192382812 nb_pixel_total : 66472 time to create 1 rle with old method : 0.07326674461364746 length of segment : 438 time for calcul the mask position with numpy : 0.0031588077545166016 nb_pixel_total : 126254 time to create 1 rle with old method : 0.1405658721923828 length of segment : 473 time for calcul the mask position with numpy : 0.0013489723205566406 nb_pixel_total : 84177 time to create 1 rle with old method : 0.09311842918395996 length of segment : 401 time for calcul the mask position with numpy : 0.0021522045135498047 nb_pixel_total : 92918 time to create 1 rle with old method : 0.10458064079284668 length of segment : 486 time for calcul the mask position with numpy : 0.0002884864807128906 nb_pixel_total : 11037 time to create 1 rle with old method : 0.012657403945922852 length of segment : 179 time for calcul the mask position with numpy : 0.000545501708984375 nb_pixel_total : 20373 time to create 1 rle with old method : 0.024981021881103516 length of segment : 219 time for calcul the mask position with numpy : 0.0005893707275390625 nb_pixel_total : 25417 time to create 1 rle with old method : 0.029036521911621094 length of segment : 296 time for calcul the mask position with numpy : 0.002336740493774414 nb_pixel_total : 97513 time to create 1 rle with old method : 0.12310075759887695 length of segment : 516 time for calcul the mask position with numpy : 0.002557992935180664 nb_pixel_total : 112600 time to create 1 rle with old method : 0.15743041038513184 length of segment : 417 time for calcul the mask position with numpy : 0.0006747245788574219 nb_pixel_total : 33299 time to create 1 rle with old method : 0.038903236389160156 length of segment : 258 time for calcul the mask position with numpy : 0.0005509853363037109 nb_pixel_total : 13969 time to create 1 rle with old method : 0.02069830894470215 length of segment : 219 time for calcul the mask position with numpy : 0.0028443336486816406 nb_pixel_total : 92531 time to create 1 rle with old method : 0.10299801826477051 length of segment : 258 time for calcul the mask position with numpy : 0.0011339187622070312 nb_pixel_total : 67578 time to create 1 rle with old method : 0.07917547225952148 length of segment : 395 time for calcul the mask position with numpy : 0.00030303001403808594 nb_pixel_total : 14429 time to create 1 rle with old method : 0.016735076904296875 length of segment : 137 time for calcul the mask position with numpy : 0.0006887912750244141 nb_pixel_total : 34731 time to create 1 rle with old method : 0.039407968521118164 length of segment : 258 time for calcul the mask position with numpy : 0.0024580955505371094 nb_pixel_total : 128075 time to create 1 rle with old method : 0.14728450775146484 length of segment : 442 time for calcul the mask position with numpy : 0.0006062984466552734 nb_pixel_total : 23383 time to create 1 rle with old method : 0.027027368545532227 length of segment : 249 time for calcul the mask position with numpy : 0.0006878376007080078 nb_pixel_total : 31585 time to create 1 rle with old method : 0.03603363037109375 length of segment : 199 time for calcul the mask position with numpy : 0.0002989768981933594 nb_pixel_total : 15713 time to create 1 rle with old method : 0.0183718204498291 length of segment : 114 time for calcul the mask position with numpy : 0.0016241073608398438 nb_pixel_total : 98402 time to create 1 rle with old method : 0.1375138759613037 length of segment : 485 time for calcul the mask position with numpy : 0.0010275840759277344 nb_pixel_total : 23946 time to create 1 rle with old method : 0.02698969841003418 length of segment : 254 time for calcul the mask position with numpy : 0.005937099456787109 nb_pixel_total : 160186 time to create 1 rle with new method : 0.010578393936157227 length of segment : 533 time for calcul the mask position with numpy : 0.0009093284606933594 nb_pixel_total : 23789 time to create 1 rle with old method : 0.02721405029296875 length of segment : 197 time for calcul the mask position with numpy : 0.002010345458984375 nb_pixel_total : 32774 time to create 1 rle with old method : 0.03783011436462402 length of segment : 370 time for calcul the mask position with numpy : 0.0009157657623291016 nb_pixel_total : 23522 time to create 1 rle with old method : 0.027161121368408203 length of segment : 167 time for calcul the mask position with numpy : 0.0005006790161132812 nb_pixel_total : 10261 time to create 1 rle with old method : 0.012441635131835938 length of segment : 118 time for calcul the mask position with numpy : 0.002229928970336914 nb_pixel_total : 45827 time to create 1 rle with old method : 0.05224275588989258 length of segment : 258 time for calcul the mask position with numpy : 0.0009136199951171875 nb_pixel_total : 21470 time to create 1 rle with old method : 0.024677038192749023 length of segment : 171 time for calcul the mask position with numpy : 0.00551295280456543 nb_pixel_total : 141282 time to create 1 rle with old method : 0.15778422355651855 length of segment : 695 time for calcul the mask position with numpy : 0.0057294368743896484 nb_pixel_total : 140876 time to create 1 rle with old method : 0.1577444076538086 length of segment : 430 time for calcul the mask position with numpy : 0.0008406639099121094 nb_pixel_total : 17102 time to create 1 rle with old method : 0.019429683685302734 length of segment : 138 time for calcul the mask position with numpy : 0.0022797584533691406 nb_pixel_total : 53961 time to create 1 rle with old method : 0.06255817413330078 length of segment : 349 time for calcul the mask position with numpy : 0.002276182174682617 nb_pixel_total : 40874 time to create 1 rle with old method : 0.04709768295288086 length of segment : 443 time for calcul the mask position with numpy : 0.0005872249603271484 nb_pixel_total : 14477 time to create 1 rle with old method : 0.01665806770324707 length of segment : 79 time for calcul the mask position with numpy : 0.00048160552978515625 nb_pixel_total : 10378 time to create 1 rle with old method : 0.012481212615966797 length of segment : 98 time for calcul the mask position with numpy : 0.0035393238067626953 nb_pixel_total : 84008 time to create 1 rle with old method : 0.09606790542602539 length of segment : 530 time for calcul the mask position with numpy : 0.0006270408630371094 nb_pixel_total : 15646 time to create 1 rle with old method : 0.01777958869934082 length of segment : 143 time for calcul the mask position with numpy : 0.00043511390686035156 nb_pixel_total : 6534 time to create 1 rle with old method : 0.007692813873291016 length of segment : 165 time for calcul the mask position with numpy : 0.001176595687866211 nb_pixel_total : 32665 time to create 1 rle with old method : 0.03798556327819824 length of segment : 255 time for calcul the mask position with numpy : 0.0013456344604492188 nb_pixel_total : 23214 time to create 1 rle with old method : 0.026573657989501953 length of segment : 292 time for calcul the mask position with numpy : 0.0006124973297119141 nb_pixel_total : 12417 time to create 1 rle with old method : 0.014620065689086914 length of segment : 124 time for calcul the mask position with numpy : 0.0014221668243408203 nb_pixel_total : 35597 time to create 1 rle with old method : 0.040630340576171875 length of segment : 298 time for calcul the mask position with numpy : 0.006088733673095703 nb_pixel_total : 157505 time to create 1 rle with new method : 0.01244497299194336 length of segment : 347 time for calcul the mask position with numpy : 0.0036995410919189453 nb_pixel_total : 125343 time to create 1 rle with old method : 0.1419520378112793 length of segment : 303 time for calcul the mask position with numpy : 0.002736806869506836 nb_pixel_total : 79463 time to create 1 rle with old method : 0.08937978744506836 length of segment : 365 time for calcul the mask position with numpy : 0.003049612045288086 nb_pixel_total : 105619 time to create 1 rle with old method : 0.12313580513000488 length of segment : 293 time for calcul the mask position with numpy : 0.0015392303466796875 nb_pixel_total : 40081 time to create 1 rle with old method : 0.04604506492614746 length of segment : 381 time for calcul the mask position with numpy : 0.0008933544158935547 nb_pixel_total : 23313 time to create 1 rle with old method : 0.02708578109741211 length of segment : 147 time for calcul the mask position with numpy : 0.0008742809295654297 nb_pixel_total : 23030 time to create 1 rle with old method : 0.02698493003845215 length of segment : 111 time for calcul the mask position with numpy : 0.002315998077392578 nb_pixel_total : 61732 time to create 1 rle with old method : 0.06974577903747559 length of segment : 317 time for calcul the mask position with numpy : 0.001592874526977539 nb_pixel_total : 26246 time to create 1 rle with old method : 0.030245065689086914 length of segment : 289 time for calcul the mask position with numpy : 0.0021109580993652344 nb_pixel_total : 74527 time to create 1 rle with old method : 0.10300207138061523 length of segment : 408 time for calcul the mask position with numpy : 0.0027315616607666016 nb_pixel_total : 69875 time to create 1 rle with old method : 0.07945108413696289 length of segment : 371 time for calcul the mask position with numpy : 0.001081228256225586 nb_pixel_total : 24492 time to create 1 rle with old method : 0.029812097549438477 length of segment : 142 time for calcul the mask position with numpy : 0.0006608963012695312 nb_pixel_total : 18950 time to create 1 rle with old method : 0.02164292335510254 length of segment : 152 time for calcul the mask position with numpy : 0.0021796226501464844 nb_pixel_total : 75956 time to create 1 rle with old method : 0.08634328842163086 length of segment : 253 time for calcul the mask position with numpy : 0.00833272933959961 nb_pixel_total : 207345 time to create 1 rle with new method : 0.014129400253295898 length of segment : 683 time for calcul the mask position with numpy : 0.0027294158935546875 nb_pixel_total : 49640 time to create 1 rle with old method : 0.05605912208557129 length of segment : 304 time for calcul the mask position with numpy : 0.0003490447998046875 nb_pixel_total : 6516 time to create 1 rle with old method : 0.007615089416503906 length of segment : 99 time for calcul the mask position with numpy : 0.001689910888671875 nb_pixel_total : 39602 time to create 1 rle with old method : 0.04530453681945801 length of segment : 171 time for calcul the mask position with numpy : 0.016842126846313477 nb_pixel_total : 300991 time to create 1 rle with new method : 0.01665782928466797 length of segment : 767 time for calcul the mask position with numpy : 0.0005705356597900391 nb_pixel_total : 12455 time to create 1 rle with old method : 0.014518260955810547 length of segment : 161 time for calcul the mask position with numpy : 0.001968860626220703 nb_pixel_total : 63879 time to create 1 rle with old method : 0.08793044090270996 length of segment : 312 time for calcul the mask position with numpy : 0.021526575088500977 nb_pixel_total : 335922 time to create 1 rle with new method : 0.28084754943847656 length of segment : 841 time for calcul the mask position with numpy : 0.00039124488830566406 nb_pixel_total : 8445 time to create 1 rle with old method : 0.010023355484008789 length of segment : 79 time for calcul the mask position with numpy : 0.0015671253204345703 nb_pixel_total : 34890 time to create 1 rle with old method : 0.03903794288635254 length of segment : 273 time for calcul the mask position with numpy : 0.0005030632019042969 nb_pixel_total : 9516 time to create 1 rle with old method : 0.01094818115234375 length of segment : 147 time for calcul the mask position with numpy : 0.002574920654296875 nb_pixel_total : 84479 time to create 1 rle with old method : 0.10027432441711426 length of segment : 364 time for calcul the mask position with numpy : 0.001409292221069336 nb_pixel_total : 34980 time to create 1 rle with old method : 0.03866171836853027 length of segment : 291 time for calcul the mask position with numpy : 0.0011472702026367188 nb_pixel_total : 30545 time to create 1 rle with old method : 0.03565645217895508 length of segment : 129 time for calcul the mask position with numpy : 0.0008833408355712891 nb_pixel_total : 16314 time to create 1 rle with old method : 0.019736528396606445 length of segment : 193 time for calcul the mask position with numpy : 0.0005145072937011719 nb_pixel_total : 13983 time to create 1 rle with old method : 0.01711869239807129 length of segment : 172 time for calcul the mask position with numpy : 0.001087188720703125 nb_pixel_total : 46819 time to create 1 rle with old method : 0.05342721939086914 length of segment : 232 time for calcul the mask position with numpy : 0.0004868507385253906 nb_pixel_total : 26424 time to create 1 rle with old method : 0.0304718017578125 length of segment : 151 time for calcul the mask position with numpy : 0.002021312713623047 nb_pixel_total : 47167 time to create 1 rle with old method : 0.05289435386657715 length of segment : 349 time for calcul the mask position with numpy : 0.003891468048095703 nb_pixel_total : 83254 time to create 1 rle with old method : 0.09380269050598145 length of segment : 406 time for calcul the mask position with numpy : 0.003501415252685547 nb_pixel_total : 56073 time to create 1 rle with old method : 0.06386303901672363 length of segment : 207 time for calcul the mask position with numpy : 0.0022864341735839844 nb_pixel_total : 61182 time to create 1 rle with old method : 0.06781363487243652 length of segment : 344 time for calcul the mask position with numpy : 0.0013878345489501953 nb_pixel_total : 32824 time to create 1 rle with old method : 0.03715658187866211 length of segment : 212 time for calcul the mask position with numpy : 0.0006766319274902344 nb_pixel_total : 18190 time to create 1 rle with old method : 0.02028346061706543 length of segment : 170 time for calcul the mask position with numpy : 0.0030760765075683594 nb_pixel_total : 62631 time to create 1 rle with old method : 0.06966161727905273 length of segment : 431 time for calcul the mask position with numpy : 0.001735687255859375 nb_pixel_total : 38533 time to create 1 rle with old method : 0.04279303550720215 length of segment : 224 time for calcul the mask position with numpy : 0.0010874271392822266 nb_pixel_total : 19999 time to create 1 rle with old method : 0.023076772689819336 length of segment : 197 time for calcul the mask position with numpy : 0.0059337615966796875 nb_pixel_total : 172481 time to create 1 rle with new method : 0.009188652038574219 length of segment : 412 time for calcul the mask position with numpy : 0.0031974315643310547 nb_pixel_total : 87665 time to create 1 rle with old method : 0.10162758827209473 length of segment : 389 time for calcul the mask position with numpy : 0.0022001266479492188 nb_pixel_total : 69192 time to create 1 rle with old method : 0.07750201225280762 length of segment : 497 time for calcul the mask position with numpy : 0.0042994022369384766 nb_pixel_total : 136137 time to create 1 rle with old method : 0.14984583854675293 length of segment : 589 time for calcul the mask position with numpy : 0.0004971027374267578 nb_pixel_total : 9322 time to create 1 rle with old method : 0.010704755783081055 length of segment : 142 time for calcul the mask position with numpy : 0.0052988529205322266 nb_pixel_total : 131936 time to create 1 rle with old method : 0.1489853858947754 length of segment : 408 time for calcul the mask position with numpy : 0.002230405807495117 nb_pixel_total : 73211 time to create 1 rle with old method : 0.08088493347167969 length of segment : 323 time for calcul the mask position with numpy : 0.004537820816040039 nb_pixel_total : 118124 time to create 1 rle with old method : 0.1413881778717041 length of segment : 507 time for calcul the mask position with numpy : 0.0005996227264404297 nb_pixel_total : 10563 time to create 1 rle with old method : 0.012174367904663086 length of segment : 109 time for calcul the mask position with numpy : 0.0006253719329833984 nb_pixel_total : 13756 time to create 1 rle with old method : 0.018921613693237305 length of segment : 72 time for calcul the mask position with numpy : 0.0019359588623046875 nb_pixel_total : 54270 time to create 1 rle with old method : 0.06424427032470703 length of segment : 155 time for calcul the mask position with numpy : 0.001338958740234375 nb_pixel_total : 32137 time to create 1 rle with old method : 0.036234140396118164 length of segment : 196 time for calcul the mask position with numpy : 0.0026247501373291016 nb_pixel_total : 63309 time to create 1 rle with old method : 0.07222247123718262 length of segment : 589 time for calcul the mask position with numpy : 0.0018389225006103516 nb_pixel_total : 50140 time to create 1 rle with old method : 0.05518627166748047 length of segment : 376 time for calcul the mask position with numpy : 0.006012678146362305 nb_pixel_total : 147141 time to create 1 rle with old method : 0.16683006286621094 length of segment : 496 time for calcul the mask position with numpy : 0.0013718605041503906 nb_pixel_total : 40608 time to create 1 rle with old method : 0.049669742584228516 length of segment : 211 time for calcul the mask position with numpy : 0.001079559326171875 nb_pixel_total : 54569 time to create 1 rle with old method : 0.06343269348144531 length of segment : 154 time for calcul the mask position with numpy : 0.0008094310760498047 nb_pixel_total : 16223 time to create 1 rle with old method : 0.02646780014038086 length of segment : 131 time for calcul the mask position with numpy : 0.00036025047302246094 nb_pixel_total : 6525 time to create 1 rle with old method : 0.008296966552734375 length of segment : 85 time for calcul the mask position with numpy : 0.001695871353149414 nb_pixel_total : 58102 time to create 1 rle with old method : 0.06495237350463867 length of segment : 357 time for calcul the mask position with numpy : 0.002477407455444336 nb_pixel_total : 88600 time to create 1 rle with old method : 0.10253071784973145 length of segment : 396 time for calcul the mask position with numpy : 0.0036170482635498047 nb_pixel_total : 80336 time to create 1 rle with old method : 0.09179496765136719 length of segment : 745 time for calcul the mask position with numpy : 0.008596181869506836 nb_pixel_total : 259084 time to create 1 rle with new method : 0.018221139907836914 length of segment : 831 time for calcul the mask position with numpy : 0.0026175975799560547 nb_pixel_total : 84517 time to create 1 rle with old method : 0.0949091911315918 length of segment : 517 time for calcul the mask position with numpy : 0.002826213836669922 nb_pixel_total : 100018 time to create 1 rle with old method : 0.11548876762390137 length of segment : 284 time for calcul the mask position with numpy : 0.0016741752624511719 nb_pixel_total : 70943 time to create 1 rle with old method : 0.08647561073303223 length of segment : 290 time for calcul the mask position with numpy : 0.0006492137908935547 nb_pixel_total : 23502 time to create 1 rle with old method : 0.04009890556335449 length of segment : 276 time for calcul the mask position with numpy : 0.0009262561798095703 nb_pixel_total : 13702 time to create 1 rle with old method : 0.01708054542541504 length of segment : 206 time for calcul the mask position with numpy : 0.0007607936859130859 nb_pixel_total : 27939 time to create 1 rle with old method : 0.032402992248535156 length of segment : 270 time for calcul the mask position with numpy : 0.0006465911865234375 nb_pixel_total : 23721 time to create 1 rle with old method : 0.027617216110229492 length of segment : 234 time for calcul the mask position with numpy : 0.0014195442199707031 nb_pixel_total : 58194 time to create 1 rle with old method : 0.06487011909484863 length of segment : 247 time for calcul the mask position with numpy : 0.0013194084167480469 nb_pixel_total : 60218 time to create 1 rle with old method : 0.06759166717529297 length of segment : 328 time for calcul the mask position with numpy : 0.0012183189392089844 nb_pixel_total : 23272 time to create 1 rle with old method : 0.026514291763305664 length of segment : 249 time for calcul the mask position with numpy : 0.00039267539978027344 nb_pixel_total : 12738 time to create 1 rle with old method : 0.015126705169677734 length of segment : 245 time for calcul the mask position with numpy : 0.002290487289428711 nb_pixel_total : 69266 time to create 1 rle with old method : 0.07876920700073242 length of segment : 474 time for calcul the mask position with numpy : 0.002315521240234375 nb_pixel_total : 113543 time to create 1 rle with old method : 0.12695908546447754 length of segment : 454 time for calcul the mask position with numpy : 0.0012176036834716797 nb_pixel_total : 39881 time to create 1 rle with old method : 0.04538130760192871 length of segment : 326 time for calcul the mask position with numpy : 0.0017027854919433594 nb_pixel_total : 68919 time to create 1 rle with old method : 0.07582235336303711 length of segment : 389 time for calcul the mask position with numpy : 0.0006780624389648438 nb_pixel_total : 25861 time to create 1 rle with old method : 0.028940677642822266 length of segment : 178 time for calcul the mask position with numpy : 0.0027604103088378906 nb_pixel_total : 81128 time to create 1 rle with old method : 0.08931660652160645 length of segment : 349 time for calcul the mask position with numpy : 0.0014603137969970703 nb_pixel_total : 57571 time to create 1 rle with old method : 0.06528615951538086 length of segment : 235 time for calcul the mask position with numpy : 0.0004291534423828125 nb_pixel_total : 10074 time to create 1 rle with old method : 0.01179647445678711 length of segment : 138 time for calcul the mask position with numpy : 0.005250453948974609 nb_pixel_total : 224499 time to create 1 rle with new method : 0.012069225311279297 length of segment : 743 time spent for convertir_results : 13.063237428665161 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 127 chid ids of type : 3594 Number RLEs to save : 39098 save missing photos in datou_result : time spend for datou_step_exec : 77.94566369056702 time spend to save output : 2.3798444271087646 total time spend for step 1 : 80.32550811767578 step2:crop_condition Mon May 19 17:51:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 7 ! batch 1 Loaded 127 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 74 About to insert : list_path_to_insert length 74 new photo from crops ! About to upload 74 photos upload in portfolio : 3736932 init cache_photo without model_param we have 74 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747669930_2825132 we have uploaded 74 photos in the portfolio 3736932 time of upload the photos Elapsed time : 17.52637553215027 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3736932 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747669952_2825132 we have uploaded 12 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.6449015140533447 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 38 About to insert : list_path_to_insert length 38 new photo from crops ! About to upload 38 photos upload in portfolio : 3736932 init cache_photo without model_param we have 38 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747669971_2825132 we have uploaded 38 photos in the portfolio 3736932 time of upload the photos Elapsed time : 9.278790473937988 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747669981_2825132 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5014541149139404 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747669983_2825132 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7005298137664795 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359373209, 1359373206, 1359373203, 1359373156, 1359373108, 1359372907, 1359372902] Looping around the photos to save general results len do output : 127 /1359438030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359438511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373209', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373206', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373203', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373156', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373108', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372907', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372902', None, None, None, None, None, '2896580') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 388 time used for this insertion : 0.029343128204345703 save_final save missing photos in datou_result : time spend for datou_step_exec : 72.12936973571777 time spend to save output : 0.03400731086730957 total time spend for step 2 : 72.16337704658508 step3:rle_unique_nms_with_priority Mon May 19 17:53:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 127 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 8.155563354492188 time for calcul the mask position with numpy : 0.5514206886291504 nb_pixel_total : 7259637 time to create 1 rle with new method : 1.0296106338500977 time for calcul the mask position with numpy : 0.04175162315368652 nb_pixel_total : 128075 time to create 1 rle with old method : 0.1407463550567627 time for calcul the mask position with numpy : 0.03986048698425293 nb_pixel_total : 34731 time to create 1 rle with old method : 0.03873491287231445 time for calcul the mask position with numpy : 0.04144644737243652 nb_pixel_total : 14429 time to create 1 rle with old method : 0.016216278076171875 time for calcul the mask position with numpy : 0.04076385498046875 nb_pixel_total : 67578 time to create 1 rle with old method : 0.08119988441467285 time for calcul the mask position with numpy : 0.047438621520996094 nb_pixel_total : 92531 time to create 1 rle with old method : 0.10838532447814941 time for calcul the mask position with numpy : 0.03927755355834961 nb_pixel_total : 13242 time to create 1 rle with old method : 0.014636516571044922 time for calcul the mask position with numpy : 0.04220867156982422 nb_pixel_total : 33299 time to create 1 rle with old method : 0.03583550453186035 time for calcul the mask position with numpy : 0.039803504943847656 nb_pixel_total : 112600 time to create 1 rle with old method : 0.12286520004272461 time for calcul the mask position with numpy : 0.04099726676940918 nb_pixel_total : 97513 time to create 1 rle with old method : 0.10794615745544434 time for calcul the mask position with numpy : 0.0343170166015625 nb_pixel_total : 25417 time to create 1 rle with old method : 0.02733898162841797 time for calcul the mask position with numpy : 0.02492833137512207 nb_pixel_total : 20373 time to create 1 rle with old method : 0.021615982055664062 time for calcul the mask position with numpy : 0.024855613708496094 nb_pixel_total : 11037 time to create 1 rle with old method : 0.011842489242553711 time for calcul the mask position with numpy : 0.025303363800048828 nb_pixel_total : 92918 time to create 1 rle with old method : 0.10362362861633301 time for calcul the mask position with numpy : 0.0260622501373291 nb_pixel_total : 84177 time to create 1 rle with old method : 0.09266448020935059 time for calcul the mask position with numpy : 0.0267636775970459 nb_pixel_total : 126254 time to create 1 rle with old method : 0.13787293434143066 time for calcul the mask position with numpy : 0.02573418617248535 nb_pixel_total : 66472 time to create 1 rle with old method : 0.07436966896057129 time for calcul the mask position with numpy : 0.02666759490966797 nb_pixel_total : 14117 time to create 1 rle with old method : 0.016140222549438477 create new chi : 3.3612091541290283 time to delete rle : 0.02403426170349121 batch 1 Loaded 35 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 13190 TO DO : save crop sub photo not yet done ! save time : 0.978302001953125 nb_obj : 26 nb_hashtags : 3 time to prepare the origin masks : 4.4268434047698975 time for calcul the mask position with numpy : 0.6151766777038574 nb_pixel_total : 7203144 time to create 1 rle with new method : 0.9093418121337891 time for calcul the mask position with numpy : 0.03293204307556152 nb_pixel_total : 10261 time to create 1 rle with old method : 0.011442184448242188 time for calcul the mask position with numpy : 0.03359794616699219 nb_pixel_total : 23214 time to create 1 rle with old method : 0.0255587100982666 time for calcul the mask position with numpy : 0.03440737724304199 nb_pixel_total : 141282 time to create 1 rle with old method : 0.15654444694519043 time for calcul the mask position with numpy : 0.03508114814758301 nb_pixel_total : 160186 time to create 1 rle with new method : 0.7549703121185303 time for calcul the mask position with numpy : 0.03373980522155762 nb_pixel_total : 23946 time to create 1 rle with old method : 0.02689838409423828 time for calcul the mask position with numpy : 0.03331732749938965 nb_pixel_total : 17102 time to create 1 rle with old method : 0.019014835357666016 time for calcul the mask position with numpy : 0.032373905181884766 nb_pixel_total : 1801 time to create 1 rle with old method : 0.0019762516021728516 time for calcul the mask position with numpy : 0.033225059509277344 nb_pixel_total : 23522 time to create 1 rle with old method : 0.02577066421508789 time for calcul the mask position with numpy : 0.03346610069274902 nb_pixel_total : 12417 time to create 1 rle with old method : 0.014015436172485352 time for calcul the mask position with numpy : 0.03466033935546875 nb_pixel_total : 98402 time to create 1 rle with old method : 0.13467860221862793 time for calcul the mask position with numpy : 0.03699088096618652 nb_pixel_total : 35597 time to create 1 rle with old method : 0.04254150390625 time for calcul the mask position with numpy : 0.034636497497558594 nb_pixel_total : 140876 time to create 1 rle with old method : 0.15856003761291504 time for calcul the mask position with numpy : 0.034491777420043945 nb_pixel_total : 31585 time to create 1 rle with old method : 0.03553199768066406 time for calcul the mask position with numpy : 0.034964561462402344 nb_pixel_total : 45827 time to create 1 rle with old method : 0.052767276763916016 time for calcul the mask position with numpy : 0.034346580505371094 nb_pixel_total : 44401 time to create 1 rle with old method : 0.04850149154663086 time for calcul the mask position with numpy : 0.03347373008728027 nb_pixel_total : 15646 time to create 1 rle with old method : 0.01767277717590332 time for calcul the mask position with numpy : 0.035971879959106445 nb_pixel_total : 32665 time to create 1 rle with old method : 0.036962032318115234 time for calcul the mask position with numpy : 0.033521413803100586 nb_pixel_total : 32774 time to create 1 rle with old method : 0.03563833236694336 time for calcul the mask position with numpy : 0.03293347358703613 nb_pixel_total : 6534 time to create 1 rle with old method : 0.007407426834106445 time for calcul the mask position with numpy : 0.034393310546875 nb_pixel_total : 84008 time to create 1 rle with old method : 0.11821532249450684 time for calcul the mask position with numpy : 0.03422045707702637 nb_pixel_total : 23383 time to create 1 rle with old method : 0.025979280471801758 time for calcul the mask position with numpy : 0.03389453887939453 nb_pixel_total : 21470 time to create 1 rle with old method : 0.02386164665222168 time for calcul the mask position with numpy : 0.033972740173339844 nb_pixel_total : 23789 time to create 1 rle with old method : 0.026392698287963867 time for calcul the mask position with numpy : 0.03423476219177246 nb_pixel_total : 15713 time to create 1 rle with old method : 0.017531871795654297 time for calcul the mask position with numpy : 0.034095048904418945 nb_pixel_total : 10378 time to create 1 rle with old method : 0.011612176895141602 time for calcul the mask position with numpy : 0.035308837890625 nb_pixel_total : 14477 time to create 1 rle with old method : 0.016185522079467773 create new chi : 4.32662558555603 time to delete rle : 0.00211334228515625 batch 1 Loaded 53 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++Number RLEs to save : 15681 TO DO : save crop sub photo not yet done ! save time : 0.9942026138305664 nb_obj : 14 nb_hashtags : 3 time to prepare the origin masks : 6.18927264213562 time for calcul the mask position with numpy : 0.5911939144134521 nb_pixel_total : 7388268 time to create 1 rle with new method : 0.7448952198028564 time for calcul the mask position with numpy : 0.045831918716430664 nb_pixel_total : 75956 time to create 1 rle with old method : 0.09775662422180176 time for calcul the mask position with numpy : 0.024293184280395508 nb_pixel_total : 18950 time to create 1 rle with old method : 0.021155595779418945 time for calcul the mask position with numpy : 0.024660348892211914 nb_pixel_total : 24492 time to create 1 rle with old method : 0.02747654914855957 time for calcul the mask position with numpy : 0.02543187141418457 nb_pixel_total : 69875 time to create 1 rle with old method : 0.07781696319580078 time for calcul the mask position with numpy : 0.02704453468322754 nb_pixel_total : 74527 time to create 1 rle with old method : 0.0847315788269043 time for calcul the mask position with numpy : 0.026314496994018555 nb_pixel_total : 26246 time to create 1 rle with old method : 0.029504776000976562 time for calcul the mask position with numpy : 0.026897668838500977 nb_pixel_total : 61732 time to create 1 rle with old method : 0.06886458396911621 time for calcul the mask position with numpy : 0.027380943298339844 nb_pixel_total : 23030 time to create 1 rle with old method : 0.026738405227661133 time for calcul the mask position with numpy : 0.024705886840820312 nb_pixel_total : 23313 time to create 1 rle with old method : 0.025208711624145508 time for calcul the mask position with numpy : 0.02638387680053711 nb_pixel_total : 40081 time to create 1 rle with old method : 0.057947635650634766 time for calcul the mask position with numpy : 0.028806209564208984 nb_pixel_total : 105619 time to create 1 rle with old method : 0.12046360969543457 time for calcul the mask position with numpy : 0.02661275863647461 nb_pixel_total : 79463 time to create 1 rle with old method : 0.08833503723144531 time for calcul the mask position with numpy : 0.02610158920288086 nb_pixel_total : 125343 time to create 1 rle with old method : 0.13923287391662598 time for calcul the mask position with numpy : 0.02743053436279297 nb_pixel_total : 157505 time to create 1 rle with new method : 0.895855188369751 create new chi : 3.550888776779175 time to delete rle : 0.0014219284057617188 batch 1 Loaded 29 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 9918 TO DO : save crop sub photo not yet done ! save time : 0.6314394474029541 nb_obj : 20 nb_hashtags : 3 time to prepare the origin masks : 8.302664041519165 time for calcul the mask position with numpy : 0.4438655376434326 nb_pixel_total : 6862649 time to create 1 rle with new method : 1.1561317443847656 time for calcul the mask position with numpy : 0.025918960571289062 nb_pixel_total : 83254 time to create 1 rle with old method : 0.09353113174438477 time for calcul the mask position with numpy : 0.026045560836791992 nb_pixel_total : 47167 time to create 1 rle with old method : 0.05276775360107422 time for calcul the mask position with numpy : 0.025176286697387695 nb_pixel_total : 5580 time to create 1 rle with old method : 0.0064923763275146484 time for calcul the mask position with numpy : 0.025336027145385742 nb_pixel_total : 46819 time to create 1 rle with old method : 0.05228757858276367 time for calcul the mask position with numpy : 0.024585962295532227 nb_pixel_total : 13412 time to create 1 rle with old method : 0.014641284942626953 time for calcul the mask position with numpy : 0.025098562240600586 nb_pixel_total : 16314 time to create 1 rle with old method : 0.01818370819091797 time for calcul the mask position with numpy : 0.02765512466430664 nb_pixel_total : 30545 time to create 1 rle with old method : 0.04971766471862793 time for calcul the mask position with numpy : 0.02776646614074707 nb_pixel_total : 34980 time to create 1 rle with old method : 0.043175697326660156 time for calcul the mask position with numpy : 0.02570033073425293 nb_pixel_total : 84479 time to create 1 rle with old method : 0.09271788597106934 time for calcul the mask position with numpy : 0.02602529525756836 nb_pixel_total : 9516 time to create 1 rle with old method : 0.010667800903320312 time for calcul the mask position with numpy : 0.02486109733581543 nb_pixel_total : 34890 time to create 1 rle with old method : 0.03962135314941406 time for calcul the mask position with numpy : 0.02611064910888672 nb_pixel_total : 8445 time to create 1 rle with old method : 0.009412765502929688 time for calcul the mask position with numpy : 0.02866196632385254 nb_pixel_total : 335922 time to create 1 rle with new method : 0.8666262626647949 time for calcul the mask position with numpy : 0.026699304580688477 nb_pixel_total : 63879 time to create 1 rle with old method : 0.07206964492797852 time for calcul the mask position with numpy : 0.025764942169189453 nb_pixel_total : 12455 time to create 1 rle with old method : 0.013768672943115234 time for calcul the mask position with numpy : 0.027331829071044922 nb_pixel_total : 300991 time to create 1 rle with new method : 0.6665327548980713 time for calcul the mask position with numpy : 0.027896881103515625 nb_pixel_total : 39602 time to create 1 rle with old method : 0.0439608097076416 time for calcul the mask position with numpy : 0.027624130249023438 nb_pixel_total : 6516 time to create 1 rle with old method : 0.0073659420013427734 time for calcul the mask position with numpy : 0.027805328369140625 nb_pixel_total : 49640 time to create 1 rle with old method : 0.05503201484680176 time for calcul the mask position with numpy : 0.027289628982543945 nb_pixel_total : 207345 time to create 1 rle with new method : 0.9278640747070312 create new chi : 5.397250413894653 time to delete rle : 0.002004861831665039 batch 1 Loaded 41 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 14256 TO DO : save crop sub photo not yet done ! save time : 0.902597188949585 nb_obj : 25 nb_hashtags : 3 time to prepare the origin masks : 11.442006587982178 time for calcul the mask position with numpy : 0.42592787742614746 nb_pixel_total : 6777808 time to create 1 rle with new method : 0.9360039234161377 time for calcul the mask position with numpy : 0.02427077293395996 nb_pixel_total : 16223 time to create 1 rle with old method : 0.017986774444580078 time for calcul the mask position with numpy : 0.04066109657287598 nb_pixel_total : 976 time to create 1 rle with old method : 0.001537322998046875 time for calcul the mask position with numpy : 0.040704965591430664 nb_pixel_total : 40608 time to create 1 rle with old method : 0.04466080665588379 time for calcul the mask position with numpy : 0.04137611389160156 nb_pixel_total : 147141 time to create 1 rle with old method : 0.16335463523864746 time for calcul the mask position with numpy : 0.040688514709472656 nb_pixel_total : 50140 time to create 1 rle with old method : 0.055573225021362305 time for calcul the mask position with numpy : 0.04076671600341797 nb_pixel_total : 63309 time to create 1 rle with old method : 0.07121658325195312 time for calcul the mask position with numpy : 0.04125213623046875 nb_pixel_total : 32137 time to create 1 rle with old method : 0.036705732345581055 time for calcul the mask position with numpy : 0.0261232852935791 nb_pixel_total : 54270 time to create 1 rle with old method : 0.060760498046875 time for calcul the mask position with numpy : 0.029928922653198242 nb_pixel_total : 13756 time to create 1 rle with old method : 0.02337956428527832 time for calcul the mask position with numpy : 0.03012537956237793 nb_pixel_total : 10563 time to create 1 rle with old method : 0.01616978645324707 time for calcul the mask position with numpy : 0.0265042781829834 nb_pixel_total : 118124 time to create 1 rle with old method : 0.13630032539367676 time for calcul the mask position with numpy : 0.02651667594909668 nb_pixel_total : 73211 time to create 1 rle with old method : 0.08090972900390625 time for calcul the mask position with numpy : 0.025485992431640625 nb_pixel_total : 131936 time to create 1 rle with old method : 0.15553689002990723 time for calcul the mask position with numpy : 0.026683807373046875 nb_pixel_total : 9322 time to create 1 rle with old method : 0.011169672012329102 time for calcul the mask position with numpy : 0.03096175193786621 nb_pixel_total : 136106 time to create 1 rle with old method : 0.17709946632385254 time for calcul the mask position with numpy : 0.02729511260986328 nb_pixel_total : 69192 time to create 1 rle with old method : 0.07704877853393555 time for calcul the mask position with numpy : 0.02586054801940918 nb_pixel_total : 87665 time to create 1 rle with old method : 0.10648918151855469 time for calcul the mask position with numpy : 0.0271146297454834 nb_pixel_total : 172481 time to create 1 rle with new method : 0.5876469612121582 time for calcul the mask position with numpy : 0.02500605583190918 nb_pixel_total : 19999 time to create 1 rle with old method : 0.022083520889282227 time for calcul the mask position with numpy : 0.024869680404663086 nb_pixel_total : 38533 time to create 1 rle with old method : 0.042289018630981445 time for calcul the mask position with numpy : 0.025566816329956055 nb_pixel_total : 62631 time to create 1 rle with old method : 0.06918883323669434 time for calcul the mask position with numpy : 0.030031442642211914 nb_pixel_total : 18190 time to create 1 rle with old method : 0.020513534545898438 time for calcul the mask position with numpy : 0.026417255401611328 nb_pixel_total : 32824 time to create 1 rle with old method : 0.03602027893066406 time for calcul the mask position with numpy : 0.02658390998840332 nb_pixel_total : 61182 time to create 1 rle with old method : 0.06955432891845703 time for calcul the mask position with numpy : 0.025672197341918945 nb_pixel_total : 56073 time to create 1 rle with old method : 0.0619046688079834 create new chi : 4.332589864730835 time to delete rle : 0.0022382736206054688 batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++Number RLEs to save : 17092 TO DO : save crop sub photo not yet done ! save time : 1.0656533241271973 nb_obj : 18 nb_hashtags : 3 time to prepare the origin masks : 9.487901210784912 time for calcul the mask position with numpy : 0.7643246650695801 nb_pixel_total : 7121160 time to create 1 rle with new method : 1.0671098232269287 time for calcul the mask position with numpy : 0.025252819061279297 nb_pixel_total : 113543 time to create 1 rle with old method : 0.12950682640075684 time for calcul the mask position with numpy : 0.02631068229675293 nb_pixel_total : 69266 time to create 1 rle with old method : 0.07712912559509277 time for calcul the mask position with numpy : 0.02466583251953125 nb_pixel_total : 12254 time to create 1 rle with old method : 0.013485193252563477 time for calcul the mask position with numpy : 0.025724411010742188 nb_pixel_total : 23272 time to create 1 rle with old method : 0.02529144287109375 time for calcul the mask position with numpy : 0.024597644805908203 nb_pixel_total : 59722 time to create 1 rle with old method : 0.06495785713195801 time for calcul the mask position with numpy : 0.023424148559570312 nb_pixel_total : 58194 time to create 1 rle with old method : 0.06178641319274902 time for calcul the mask position with numpy : 0.024898529052734375 nb_pixel_total : 23721 time to create 1 rle with old method : 0.025387287139892578 time for calcul the mask position with numpy : 0.024775266647338867 nb_pixel_total : 27939 time to create 1 rle with old method : 0.030462980270385742 time for calcul the mask position with numpy : 0.02426433563232422 nb_pixel_total : 13702 time to create 1 rle with old method : 0.01500248908996582 time for calcul the mask position with numpy : 0.023844480514526367 nb_pixel_total : 23502 time to create 1 rle with old method : 0.025517702102661133 time for calcul the mask position with numpy : 0.024941205978393555 nb_pixel_total : 70943 time to create 1 rle with old method : 0.07683944702148438 time for calcul the mask position with numpy : 0.02460956573486328 nb_pixel_total : 100018 time to create 1 rle with old method : 0.11130952835083008 time for calcul the mask position with numpy : 0.025037765502929688 nb_pixel_total : 84517 time to create 1 rle with old method : 0.09257006645202637 time for calcul the mask position with numpy : 0.02718329429626465 nb_pixel_total : 259084 time to create 1 rle with new method : 0.8442728519439697 time for calcul the mask position with numpy : 0.025750160217285156 nb_pixel_total : 80336 time to create 1 rle with old method : 0.08965492248535156 time for calcul the mask position with numpy : 0.02569890022277832 nb_pixel_total : 88600 time to create 1 rle with old method : 0.0970313549041748 time for calcul the mask position with numpy : 0.024819612503051758 nb_pixel_total : 58102 time to create 1 rle with old method : 0.06301760673522949 time for calcul the mask position with numpy : 0.02523016929626465 nb_pixel_total : 6525 time to create 1 rle with old method : 0.007127285003662109 create new chi : 4.203435659408569 time to delete rle : 0.0030236244201660156 batch 1 Loaded 37 chid ids of type : 3594 ++++++++++++++++++++++Number RLEs to save : 15074 TO DO : save crop sub photo not yet done ! save time : 0.9339637756347656 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 5.035429239273071 time for calcul the mask position with numpy : 1.1438438892364502 nb_pixel_total : 7786467 time to create 1 rle with new method : 0.9589269161224365 time for calcul the mask position with numpy : 0.025606632232666016 nb_pixel_total : 224499 time to create 1 rle with new method : 0.6262013912200928 time for calcul the mask position with numpy : 0.026439189910888672 nb_pixel_total : 10074 time to create 1 rle with old method : 0.011138200759887695 time for calcul the mask position with numpy : 0.026194095611572266 nb_pixel_total : 57571 time to create 1 rle with old method : 0.06514453887939453 time for calcul the mask position with numpy : 0.027645587921142578 nb_pixel_total : 81128 time to create 1 rle with old method : 0.09064912796020508 time for calcul the mask position with numpy : 0.024794578552246094 nb_pixel_total : 25861 time to create 1 rle with old method : 0.02998495101928711 time for calcul the mask position with numpy : 0.026291608810424805 nb_pixel_total : 68919 time to create 1 rle with old method : 0.07628393173217773 time for calcul the mask position with numpy : 0.029050111770629883 nb_pixel_total : 39881 time to create 1 rle with old method : 0.04464459419250488 create new chi : 3.3122498989105225 time to delete rle : 0.0010256767272949219 batch 1 Loaded 15 chid ids of type : 3594 ++++++++++Number RLEs to save : 6876 TO DO : save crop sub photo not yet done ! save time : 0.4545753002166748 map_output_result : {1359373209: (0.0, 'Should be the crop_list due to order', 0), 1359373206: (0.0, 'Should be the crop_list due to order', 0), 1359373203: (0.0, 'Should be the crop_list due to order', 0), 1359373156: (0.0, 'Should be the crop_list due to order', 0), 1359373108: (0.0, 'Should be the crop_list due to order', 0), 1359372907: (0.0, 'Should be the crop_list due to order', 0), 1359372902: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359373209, 1359373206, 1359373203, 1359373156, 1359373108, 1359372907, 1359372902] Looping around the photos to save general results len do output : 7 /1359373209.Didn't retrieve data . /1359373206.Didn't retrieve data . /1359373203.Didn't retrieve data . /1359373156.Didn't retrieve data . /1359373108.Didn't retrieve data . /1359372907.Didn't retrieve data . /1359372902.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373209', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373206', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373203', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373156', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373108', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372907', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372902', None, None, None, None, None, '2896580') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.01287698745727539 save_final save missing photos in datou_result : time spend for datou_step_exec : 88.46144127845764 time spend to save output : 0.013225317001342773 total time spend for step 3 : 88.47466659545898 step4:ventilate_hashtags_in_portfolio Mon May 19 17:54:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23127827 get user id for portfolio 23127827 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23127827 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_fonce','papier','background','autre','flou','mal_croppe','metal','carton','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23127827 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_fonce','papier','background','autre','flou','mal_croppe','metal','carton','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23127827 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','pet_fonce','papier','background','autre','flou','mal_croppe','metal','carton','pet_clair','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23128660,23128661,23128662,23128663,23128664,23128665,23128666,23128667,23128668,23128669,23128670?tags=pehd,pet_fonce,papier,background,autre,flou,mal_croppe,metal,carton,pet_clair,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359373209, 1359373206, 1359373203, 1359373156, 1359373108, 1359372907, 1359372902] Looping around the photos to save general results len do output : 1 /23127827. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373209', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373206', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373203', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373156', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373108', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372907', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372902', None, None, None, None, None, '2896580') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.015764713287353516 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5912525653839111 time spend to save output : 0.016039371490478516 total time spend for step 4 : 1.6072919368743896 step5:final Mon May 19 17:54:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359373209: ('0.1319593770667989',), 1359373206: ('0.1319593770667989',), 1359373203: ('0.1319593770667989',), 1359373156: ('0.1319593770667989',), 1359373108: ('0.1319593770667989',), 1359372907: ('0.1319593770667989',), 1359372902: ('0.1319593770667989',)} new output for save of step final : {1359373209: ('0.1319593770667989',), 1359373206: ('0.1319593770667989',), 1359373203: ('0.1319593770667989',), 1359373156: ('0.1319593770667989',), 1359373108: ('0.1319593770667989',), 1359372907: ('0.1319593770667989',), 1359372902: ('0.1319593770667989',)} [1359373209, 1359373206, 1359373203, 1359373156, 1359373108, 1359372907, 1359372902] Looping around the photos to save general results len do output : 7 /1359373209.Didn't retrieve data . /1359373206.Didn't retrieve data . /1359373203.Didn't retrieve data . /1359373156.Didn't retrieve data . /1359373108.Didn't retrieve data . /1359372907.Didn't retrieve data . /1359372902.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373209', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373206', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373203', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373156', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373108', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372907', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372902', None, None, None, None, None, '2896580') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.012492179870605469 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10689711570739746 time spend to save output : 0.01293635368347168 total time spend for step 5 : 0.11983346939086914 step6:blur_detection Mon May 19 17:54:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82.jpg resize: (2160, 3840) 1359373209 -5.723364406878143 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a.jpg resize: (2160, 3840) 1359373206 -5.774880926230202 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759.jpg resize: (2160, 3840) 1359373203 -5.485202924834315 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104.jpg resize: (2160, 3840) 1359373156 -5.73871606898229 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9.jpg resize: (2160, 3840) 1359373108 -5.7565580646425 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10.jpg resize: (2160, 3840) 1359372907 -5.728750239627158 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71.jpg resize: (2160, 3840) 1359372902 -5.684375607615633 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160327_0.png resize: (405, 536) 1359438030 -3.3055107696679613 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160319_0.png resize: (514, 221) 1359438032 -0.6785827246096273 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160322_0.png resize: (225, 170) 1359438033 -2.5992544609419306 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160315_0.png resize: (413, 386) 1359438034 -4.547026547957796 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160320_0.png resize: (313, 748) 1359438036 -2.0546497328621953 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160321_0.png resize: (245, 218) 1359438037 -2.364098600362897 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160312_0.png resize: (385, 286) 1359438038 -3.234211228923649 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160317_0.png resize: (218, 128) 1359438039 -3.221739842038872 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160350_0.png resize: (251, 193) 1359438041 -4.578131091636139 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160335_0.png resize: (275, 269) 1359438042 -2.2863278504361313 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160351_0.png resize: (261, 164) 1359438043 -2.079755702321802 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160343_0.png resize: (331, 277) 1359438045 -3.700877238626119 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160345_0.png resize: (79, 240) 1359438046 -1.3724712542235271 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160329_0.png resize: (197, 267) 1359438047 -0.30937829933048705 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160347_0.png resize: (493, 244) 1359438048 -4.345912124242759 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160346_0.png resize: (97, 166) 1359438050 -3.8348946988219548 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160328_0.png resize: (249, 232) 1359438051 0.35204569463912255 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160348_0.png resize: (129, 149) 1359438052 -3.2825173303982984 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160344_0.png resize: (286, 256) 1359438054 -0.6860322197084291 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160332_0.png resize: (254, 125) 1359438055 -2.9787804569467125 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160352_0.png resize: (120, 132) 1359438056 -2.6938985110267053 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160338_0.png resize: (258, 315) 1359438058 -1.571919680163354 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160336_0.png resize: (166, 182) 1359438059 -2.3564660411321725 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160330_0.png resize: (114, 205) 1359438060 -1.576677775549496 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160331_0.png resize: (472, 305) 1359438061 -1.4313036213972463 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160334_0.png resize: (159, 191) 1359438063 -3.3358957969523346 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160349_0.png resize: (164, 51) 1359438064 -3.3653109901159244 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160358_0.png resize: (374, 146) 1359438065 -2.446190543920496 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160360_0.png resize: (110, 339) 1359438067 -1.936444528945629 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160365_0.png resize: (128, 331) 1359438068 -2.770695373321399 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160362_0.png resize: (289, 200) 1359438069 -3.2384994858955167 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160359_0.png resize: (140, 192) 1359438072 -1.6446874783408212 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160363_0.png resize: (380, 300) 1359438073 -3.2757814772515887 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160383_0.png resize: (168, 103) 1359438074 -2.289760211770069 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160378_0.png resize: (147, 92) 1359438075 1.4397616129876083 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160373_0.png resize: (161, 130) 1359438077 -2.2236834388382785 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160372_0.png resize: (694, 556) 1359438078 -4.281669761852752 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160369_0.png resize: (317, 286) 1359438079 -0.3879744774772572 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160385_0.png resize: (132, 279) 1359438080 -0.7101437235786698 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160377_0.png resize: (271, 207) 1359438082 -4.248562734924621 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160371_0.png resize: (148, 383) 1359438083 -1.2986497392552196 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160387_0.png resize: (402, 393) 1359438084 -2.5152334978121034 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160380_0.png resize: (286, 158) 1359438086 -3.5559923949518 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160376_0.png resize: (77, 160) 1359438087 -3.5169641791898276 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160370_0.png resize: (99, 86) 1359438088 -2.2234387162145777 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160409_0.png resize: (465, 479) 1359438090 -4.470633030202354 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160406_0.png resize: (193, 213) 1359438091 -1.436411624832301 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160410_0.png resize: (207, 205) 1359438092 5.33456910213137 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160389_0.png resize: (344, 244) 1359438094 -2.9041529663799976 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160390_0.png resize: (183, 247) 1359438095 -2.881266360733918 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160397_0.png resize: (434, 255) 1359438096 -4.037237320050956 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160398_0.png resize: (586, 305) 1359438098 -4.372055444375178 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160391_0.png resize: (169, 155) 1359438099 -2.903347192245949 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160394_0.png resize: (195, 165) 1359438100 -2.164294139191325 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160405_0.png resize: (150, 482) 1359438101 -3.6731424484730586 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160412_0.png resize: (121, 182) 1359438103 -1.1810622605268462 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160399_0.png resize: (142, 91) 1359438104 -3.127298329997374 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160407_0.png resize: (545, 165) 1359438105 -2.6694065750456395 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160417_0.png resize: (795, 562) 1359438107 -2.0203819918609103 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160416_0.png resize: (662, 163) 1359438108 -4.522912432792015 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160428_0.png resize: (240, 151) 1359438109 4.2645629611547395 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160421_0.png resize: (276, 132) 1359438110 -2.221850495545137 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160425_0.png resize: (242, 440) 1359438112 -0.5760793401860069 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160419_0.png resize: (275, 477) 1359438113 -2.012068197709559 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160423_0.png resize: (267, 200) 1359438114 -1.0290205674968722 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160422_0.png resize: (203, 122) 1359438116 -2.3553987075543574 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160429_0.png resize: (398, 434) 1359438117 -3.2462097689542344 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160427_0.png resize: (248, 155) 1359438118 -3.3240732465420972 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160426_0.png resize: (262, 329) 1359438120 -4.183822912969263 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160413_0.png resize: (85, 114) 1359438121 -0.5941308100013837 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160420_0.png resize: (282, 346) 1359438122 -1.6637012011693746 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160424_0.png resize: (234, 175) 1359438123 -3.3896061052193964 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160436_0.png resize: (138, 96) 1359438125 -2.90370382475048 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160437_0.png resize: (682, 506) 1359438126 -4.270435277763919 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160325_0.png resize: (137, 161) 1359438198 -1.7231894932966687 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160311_0.png resize: (123, 180) 1359438199 -2.5235706541037404 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160316_0.png resize: (179, 83) 1359438200 -3.202840019399458 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160355_0.png resize: (303, 553) 1359438202 -2.870051577753454 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160382_0.png resize: (193, 170) 1359438203 -2.679752050917907 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160403_0.png resize: (100, 165) 1359438204 -2.1910477946773814 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160404_0.png resize: (72, 241) 1359438206 -2.883241014351404 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160392_0.png resize: (430, 278) 1359438207 -3.139213223641249 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160411_0.png resize: (150, 505) 1359438208 -3.761993581196597 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160393_0.png resize: (223, 249) 1359438210 -4.285915749911162 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160433_0.png resize: (174, 198) 1359438211 -1.5360436077021034 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160431_0.png resize: (298, 210) 1359438212 -4.878452543992755 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160318_0.png resize: (286, 106) 1359438434 -3.206316553021565 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160323_0.png resize: (213, 693) 1359438435 -3.3736732622887544 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160326_0.png resize: (257, 172) 1359438436 -3.667076910260633 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160313_0.png resize: (466, 392) 1359438437 -3.2810254901804115 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160314_0.png resize: (336, 337) 1359438439 -4.534594097303655 treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160324_0.png resize: (395, 247) 1359438440 -3.828254340023839 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160333_0.png resize: (514, 440) 1359438442 -2.7601588942528994 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160341_0.png resize: (430, 404) 1359438444 -3.718387920695689 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160353_0.png resize: (288, 174) 1359438445 -3.21590801921913 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160337_0.png resize: (96, 152) 1359438446 -3.129947996730518 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160340_0.png resize: (632, 322) 1359438447 -3.517592786408903 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160356_0.png resize: (365, 297) 1359438449 -3.727416205300962 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160364_0.png resize: (370, 300) 1359438450 -2.741139564052424 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160354_0.png resize: (326, 822) 1359438451 -2.7130343456127717 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160357_0.png resize: (288, 540) 1359438453 -2.76499973370864 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160367_0.png resize: (235, 450) 1359438454 -3.155949354390829 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160361_0.png resize: (310, 291) 1359438455 -4.68466622265567 treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160366_0.png resize: (151, 171) 1359438456 -1.7913629833529345 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160374_0.png resize: (298, 253) 1359438458 -4.035826943882472 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160368_0.png resize: (624, 467) 1359438459 -4.57461917913692 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160379_0.png resize: (355, 322) 1359438460 -2.957879683444459 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160381_0.png resize: (123, 281) 1359438462 -1.5600409399940518 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160384_0.png resize: (216, 256) 1359438463 -4.743077653303746 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160375_0.png resize: (818, 550) 1359438464 -4.010655103198656 treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160386_0.png resize: (281, 232) 1359438466 -3.432189232515018 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160408_0.png resize: (374, 154) 1359438467 -3.8157462905392294 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160400_0.png resize: (392, 516) 1359438469 -3.827083480420423 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160402_0.png resize: (461, 405) 1359438470 -3.6194792364217863 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160388_0.png resize: (196, 537) 1359438471 -2.51944765893735 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160401_0.png resize: (317, 272) 1359438472 -3.6591104752891894 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160396_0.png resize: (284, 544) 1359438474 -3.2332583945224975 treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160395_0.png resize: (372, 613) 1359438475 -4.197023579341976 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160415_0.png resize: (390, 298) 1359438476 -3.306257747523369 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160418_0.png resize: (514, 212) 1359438478 -4.415361733806178 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160414_0.png resize: (354, 199) 1359438479 -3.332786140516959 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160434_0.png resize: (341, 439) 1359438480 -4.3240348442327505 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160435_0.png resize: (229, 354) 1359438481 -3.5790614695229963 treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160432_0.png resize: (380, 224) 1359438483 -4.552681555228362 treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160430_0.png resize: (454, 357) 1359438496 -2.1543924459885146 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160339_0.png resize: (170, 152) 1359438509 -3.8573143926133038 treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160342_0.png resize: (137, 155) 1359438511 -3.346062124811391 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 134 time used for this insertion : 0.016864299774169922 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 134 time used for this insertion : 0.028778076171875 save missing photos in datou_result : time spend for datou_step_exec : 33.18145203590393 time spend to save output : 0.05161643028259277 total time spend for step 6 : 33.23306846618652 step7:brightness Mon May 19 17:55:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82.jpg treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a.jpg treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759.jpg treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104.jpg treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9.jpg treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10.jpg treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71.jpg treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160327_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160319_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160322_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160315_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160320_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160321_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160312_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160317_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160350_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160335_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160351_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160343_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160345_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160329_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160347_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160346_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160328_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160348_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160344_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160332_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160352_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160338_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160336_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160330_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160331_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160334_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160349_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160358_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160360_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160365_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160362_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160359_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160363_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160383_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160378_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160373_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160372_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160369_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160385_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160377_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160371_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160387_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160380_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160376_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160370_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160409_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160406_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160410_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160389_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160390_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160397_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160398_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160391_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160394_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160405_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160412_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160399_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160407_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160417_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160416_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160428_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160421_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160425_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160419_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160423_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160422_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160429_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160427_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160426_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160413_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160420_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160424_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160436_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160437_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160325_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160311_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160316_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160355_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160382_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160403_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160404_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160392_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160411_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160393_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160433_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160431_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160318_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160323_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160326_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160313_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160314_0.png treat image : temp/1747669829_2825132_1359373209_31eb72108ba461608f046739b6264f82_rle_crop_3804160324_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160333_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160341_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160353_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160337_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160340_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160356_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160364_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160354_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160357_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160367_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160361_0.png treat image : temp/1747669829_2825132_1359373203_39ae214c82d19eafe1e380d496e6f759_rle_crop_3804160366_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160374_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160368_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160379_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160381_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160384_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160375_0.png treat image : temp/1747669829_2825132_1359373156_5c2c65ad76369a7afc155fe756fa5104_rle_crop_3804160386_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160408_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160400_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160402_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160388_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160401_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160396_0.png treat image : temp/1747669829_2825132_1359373108_fba667220f927f102979ddf99c193ba9_rle_crop_3804160395_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160415_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160418_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160414_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160434_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160435_0.png treat image : temp/1747669829_2825132_1359372902_446cb05b41da78d6261e0f67924a9d71_rle_crop_3804160432_0.png treat image : temp/1747669829_2825132_1359372907_603c49fc313dc9c1e3655f164b40fc10_rle_crop_3804160430_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160339_0.png treat image : temp/1747669829_2825132_1359373206_68e6b8e84f18a6224143bd3afdb9711a_rle_crop_3804160342_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 134 time used for this insertion : 0.016563892364501953 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 134 time used for this insertion : 0.028731584548950195 save missing photos in datou_result : time spend for datou_step_exec : 7.833238363265991 time spend to save output : 0.0506746768951416 total time spend for step 7 : 7.883913040161133 step8:velours_tree Mon May 19 17:55:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.13240671157836914 time spend to save output : 5.53131103515625e-05 total time spend for step 8 : 0.1324620246887207 step9:send_mail_cod Mon May 19 17:55:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23127827_19-05-2025_17_55_15.pdf 23128660 change filename to text .imagette231286601747670115 23128661 change filename to text .change filename to text .imagette231286611747670115 23128662 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231286621747670115 23128663 imagette231286631747670116 23128664 imagette231286641747670116 23128665 imagette231286651747670116 23128666 imagette231286661747670116 23128667 imagette231286671747670116 23128668 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231286681747670116 23128669 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231286691747670118 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23127827 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23128660,23128661,23128662,23128663,23128664,23128665,23128666,23128667,23128668,23128669,23128670?tags=pehd,pet_fonce,papier,background,autre,flou,mal_croppe,metal,carton,pet_clair,environnement args[1359373209] : ((1359373209, -5.723364406878143, 492609224), (1359373209, 0.743365885118686, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com args[1359373206] : ((1359373206, -5.774880926230202, 492609224), (1359373206, 0.7419317220575132, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com args[1359373203] : ((1359373203, -5.485202924834315, 492609224), (1359373203, 0.5995572696584812, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com args[1359373156] : ((1359373156, -5.73871606898229, 492609224), (1359373156, 0.7771644135195508, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com args[1359373108] : ((1359373108, -5.7565580646425, 492609224), (1359373108, 0.578433371611948, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com args[1359372907] : ((1359372907, -5.728750239627158, 492609224), (1359372907, 0.9442941778444437, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com args[1359372902] : ((1359372902, -5.684375607615633, 492609224), (1359372902, 0.2668963491140414, 2107752395), '0.1319593770667989') We are sending mail with results at report@fotonower.com refus_total : 0.1319593770667989 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23127827 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23127827_19-05-2025_17_55_15.pdf results_Auto_P23127827_19-05-2025_17_55_15.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23127827_19-05-2025_17_55_15.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23127827','results_Auto_P23127827_19-05-2025_17_55_15.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23127827_19-05-2025_17_55_15.pdf','pdf','','0.7','0.1319593770667989') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23127827

https://www.fotonower.com/image?json=false&list_photos_id=1359373209
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359373206
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359373203
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359373156
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359373108
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359372907
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359372902
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 13.20%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pehd: https://www.fotonower.com/view/23128660?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23128661?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23128662?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23128668?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23128669?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23127827_19-05-2025_17_55_15.pdf.

Lien vers velours :https://www.fotonower.com/velours/23128660,23128661,23128662,23128663,23128664,23128665,23128666,23128667,23128668,23128669,23128670?tags=pehd,pet_fonce,papier,background,autre,flou,mal_croppe,metal,carton,pet_clair,environnement.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 19 May 2025 15:55:23 GMT Content-Length: 0 Connection: close X-Message-Id: X8sYlYRdQU-xLpmpxj9kpQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359373209, 1359373206, 1359373203, 1359373156, 1359373108, 1359372907, 1359372902] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373209', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373206', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373203', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373156', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373108', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372907', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372902', None, None, None, None, None, '2896580') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.01565861701965332 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.710130214691162 time spend to save output : 0.015898942947387695 total time spend for step 9 : 8.72602915763855 step10:split_time_score Mon May 19 17:55:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 7),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 19052025 23127827 Nombre de photos uploadées : 7 / 23040 (0%) 19052025 23127827 Nombre de photos taguées (types de déchets): 0 / 7 (0%) 19052025 23127827 Nombre de photos taguées (volume) : 0 / 7 (0%) elapsed_time : load_data_split_time_score 2.86102294921875e-06 elapsed_time : order_list_meta_photo_and_scores 9.5367431640625e-06 ??????? elapsed_time : fill_and_build_computed_from_old_data 0.0005142688751220703 elapsed_time : insert_dashboard_record_day_entry 0.02443981170654297 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.12806391095562103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23104924_19-05-2025_10_08_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23104924 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23104924 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23104935 order by id desc limit 1 Qualite : 0.11080586682800225 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23106802_19-05-2025_11_02_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23106802 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23106802 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23110969 order by id desc limit 1 Qualite : 0.10507966553069274 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23110971_19-05-2025_12_21_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23110971 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23110971 AND mptpi.`type`=3594 To do Qualite : 0.18118987911522633 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118015_19-05-2025_14_45_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118015 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118015 AND mptpi.`type`=3594 To do Qualite : 0.13490794676941195 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23118018_19-05-2025_15_04_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23118018 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23118018 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23127824 order by id desc limit 1 Qualite : 0.1319593770667989 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23127827_19-05-2025_17_55_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23127827 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23127827 AND mptpi.`type`=3594 To do Qualite : 0.18142057442370754 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23122127_19-05-2025_16_24_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23122127 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23122127 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23122145 order by id desc limit 1 Qualite : 0.13344639331427013 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23124150_19-05-2025_16_51_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23124150 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23124150 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23127829 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'19052025': {'nb_upload': 7, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359373209, 1359373206, 1359373203, 1359373156, 1359373108, 1359372907, 1359372902] Looping around the photos to save general results len do output : 1 /23127827Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373209', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373206', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373203', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373156', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359373108', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372907', None, None, None, None, None, '2896580') ('3318', None, None, None, None, None, None, None, '2896580') ('3318', '23127827', '1359372902', None, None, None, None, None, '2896580') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.013574600219726562 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.973992347717285 time spend to save output : 0.013814687728881836 total time spend for step 10 : 4.987807035446167 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 7 set_done_treatment 140.38user 114.27system 5:03.58elapsed 83%CPU (0avgtext+0avgdata 5338240maxresident)k 516744inputs+126520outputs (106major+13084915minor)pagefaults 0swaps