python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3148537 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2909932'] with mtr_portfolio_ids : ['23171910'] and first list_photo_ids : [] new path : /proc/3148537/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 4 ; length of list_pids : 4 ; length of list_args : 4 time to download the photos : 0.7100205421447754 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue May 20 17:00:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-20 17:00:34.424137: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-20 17:00:34.455171: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-20 17:00:34.457289: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7feda0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-20 17:00:34.457356: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-20 17:00:34.461821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-20 17:00:34.755583: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x42efa9f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-20 17:00:34.755660: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-20 17:00:34.758880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-20 17:00:34.759445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 17:00:34.762965: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 17:00:34.766312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-20 17:00:34.766892: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-20 17:00:34.769948: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-20 17:00:34.771451: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-20 17:00:34.777277: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-20 17:00:34.778772: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-20 17:00:34.778861: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 17:00:34.779858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-20 17:00:34.779877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-20 17:00:34.779887: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-20 17:00:34.781210: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9984 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-20 17:00:35.097907: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-20 17:00:35.098004: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 17:00:35.098027: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 17:00:35.098047: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-20 17:00:35.098066: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-20 17:00:35.098085: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-20 17:00:35.098104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-20 17:00:35.098123: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-20 17:00:35.099606: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-20 17:00:35.100949: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-20 17:00:35.101003: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-20 17:00:35.101028: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 17:00:35.101051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-20 17:00:35.101073: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-20 17:00:35.101095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-20 17:00:35.101117: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-20 17:00:35.101140: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-20 17:00:35.102587: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-20 17:00:35.102617: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-20 17:00:35.102628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-20 17:00:35.102637: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-20 17:00:35.104230: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9984 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-20 17:00:45.985754: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-20 17:00:46.194152: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 40 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 40 Detection mask done ! Trying to reset tf kernel 3149088 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 555 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5844 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.02091836929321289 nb_pixel_total : 389138 time to create 1 rle with new method : 0.03284811973571777 length of segment : 496 time for calcul the mask position with numpy : 0.0002052783966064453 nb_pixel_total : 8037 time to create 1 rle with old method : 0.009664535522460938 length of segment : 99 time for calcul the mask position with numpy : 0.001089334487915039 nb_pixel_total : 72334 time to create 1 rle with old method : 0.08086800575256348 length of segment : 345 time for calcul the mask position with numpy : 0.0008401870727539062 nb_pixel_total : 55532 time to create 1 rle with old method : 0.06232619285583496 length of segment : 289 time for calcul the mask position with numpy : 0.0005843639373779297 nb_pixel_total : 34959 time to create 1 rle with old method : 0.040144920349121094 length of segment : 256 time for calcul the mask position with numpy : 0.0015535354614257812 nb_pixel_total : 100091 time to create 1 rle with old method : 0.11204028129577637 length of segment : 420 time for calcul the mask position with numpy : 0.0007586479187011719 nb_pixel_total : 36647 time to create 1 rle with old method : 0.04205441474914551 length of segment : 249 time for calcul the mask position with numpy : 0.0003552436828613281 nb_pixel_total : 19892 time to create 1 rle with old method : 0.023275375366210938 length of segment : 160 time for calcul the mask position with numpy : 0.0006611347198486328 nb_pixel_total : 33080 time to create 1 rle with old method : 0.04223179817199707 length of segment : 230 time for calcul the mask position with numpy : 0.00031065940856933594 nb_pixel_total : 6449 time to create 1 rle with old method : 0.007782697677612305 length of segment : 86 time for calcul the mask position with numpy : 0.000423431396484375 nb_pixel_total : 11380 time to create 1 rle with old method : 0.013745784759521484 length of segment : 98 time for calcul the mask position with numpy : 0.001058340072631836 nb_pixel_total : 27154 time to create 1 rle with old method : 0.039180755615234375 length of segment : 239 time for calcul the mask position with numpy : 0.001605987548828125 nb_pixel_total : 65470 time to create 1 rle with old method : 0.07693672180175781 length of segment : 354 time for calcul the mask position with numpy : 0.00020623207092285156 nb_pixel_total : 7642 time to create 1 rle with old method : 0.009236574172973633 length of segment : 77 time for calcul the mask position with numpy : 0.0013034343719482422 nb_pixel_total : 40785 time to create 1 rle with old method : 0.05553579330444336 length of segment : 192 time for calcul the mask position with numpy : 0.0012786388397216797 nb_pixel_total : 31101 time to create 1 rle with old method : 0.037206172943115234 length of segment : 301 time for calcul the mask position with numpy : 0.0018525123596191406 nb_pixel_total : 32633 time to create 1 rle with old method : 0.038101911544799805 length of segment : 249 time for calcul the mask position with numpy : 0.0022461414337158203 nb_pixel_total : 90184 time to create 1 rle with old method : 0.12802815437316895 length of segment : 301 time for calcul the mask position with numpy : 0.007920980453491211 nb_pixel_total : 133944 time to create 1 rle with old method : 0.15222930908203125 length of segment : 382 time for calcul the mask position with numpy : 0.0007114410400390625 nb_pixel_total : 21554 time to create 1 rle with old method : 0.025582313537597656 length of segment : 114 time for calcul the mask position with numpy : 0.0048024654388427734 nb_pixel_total : 81754 time to create 1 rle with old method : 0.09625053405761719 length of segment : 341 time for calcul the mask position with numpy : 0.004644632339477539 nb_pixel_total : 87892 time to create 1 rle with old method : 0.10416817665100098 length of segment : 362 time for calcul the mask position with numpy : 0.003554821014404297 nb_pixel_total : 75450 time to create 1 rle with old method : 0.10738301277160645 length of segment : 330 time for calcul the mask position with numpy : 0.0009186267852783203 nb_pixel_total : 28269 time to create 1 rle with old method : 0.033182382583618164 length of segment : 234 time for calcul the mask position with numpy : 0.014364480972290039 nb_pixel_total : 283643 time to create 1 rle with new method : 0.03648257255554199 length of segment : 432 time for calcul the mask position with numpy : 0.0009286403656005859 nb_pixel_total : 16711 time to create 1 rle with old method : 0.019388675689697266 length of segment : 219 time for calcul the mask position with numpy : 0.0017726421356201172 nb_pixel_total : 33483 time to create 1 rle with old method : 0.038717031478881836 length of segment : 272 time for calcul the mask position with numpy : 0.0013365745544433594 nb_pixel_total : 54040 time to create 1 rle with old method : 0.06422615051269531 length of segment : 403 time for calcul the mask position with numpy : 0.0014312267303466797 nb_pixel_total : 31462 time to create 1 rle with old method : 0.05279040336608887 length of segment : 171 time for calcul the mask position with numpy : 0.0015895366668701172 nb_pixel_total : 30626 time to create 1 rle with old method : 0.03529548645019531 length of segment : 205 time for calcul the mask position with numpy : 0.0034432411193847656 nb_pixel_total : 73817 time to create 1 rle with old method : 0.08370566368103027 length of segment : 324 time for calcul the mask position with numpy : 0.0009891986846923828 nb_pixel_total : 31890 time to create 1 rle with old method : 0.03807806968688965 length of segment : 175 time for calcul the mask position with numpy : 0.0012319087982177734 nb_pixel_total : 37802 time to create 1 rle with old method : 0.04359626770019531 length of segment : 166 time for calcul the mask position with numpy : 0.00043487548828125 nb_pixel_total : 9913 time to create 1 rle with old method : 0.011945009231567383 length of segment : 92 time for calcul the mask position with numpy : 0.0064792633056640625 nb_pixel_total : 110388 time to create 1 rle with old method : 0.12541723251342773 length of segment : 451 time for calcul the mask position with numpy : 0.0018320083618164062 nb_pixel_total : 36711 time to create 1 rle with old method : 0.04340195655822754 length of segment : 302 time for calcul the mask position with numpy : 0.0006060600280761719 nb_pixel_total : 10112 time to create 1 rle with old method : 0.012169599533081055 length of segment : 141 time for calcul the mask position with numpy : 0.0007145404815673828 nb_pixel_total : 18503 time to create 1 rle with old method : 0.02159428596496582 length of segment : 151 time for calcul the mask position with numpy : 0.0018515586853027344 nb_pixel_total : 51405 time to create 1 rle with old method : 0.05926918983459473 length of segment : 210 time for calcul the mask position with numpy : 0.0025281906127929688 nb_pixel_total : 58204 time to create 1 rle with old method : 0.06972002983093262 length of segment : 246 time for calcul the mask position with numpy : 0.0048825740814208984 nb_pixel_total : 122217 time to create 1 rle with old method : 0.1387009620666504 length of segment : 499 time for calcul the mask position with numpy : 0.00089263916015625 nb_pixel_total : 23546 time to create 1 rle with old method : 0.027324676513671875 length of segment : 203 time for calcul the mask position with numpy : 0.002260446548461914 nb_pixel_total : 47893 time to create 1 rle with old method : 0.05578351020812988 length of segment : 450 time for calcul the mask position with numpy : 0.0010671615600585938 nb_pixel_total : 32069 time to create 1 rle with old method : 0.03742218017578125 length of segment : 176 time for calcul the mask position with numpy : 0.0034761428833007812 nb_pixel_total : 65828 time to create 1 rle with old method : 0.0767676830291748 length of segment : 329 time for calcul the mask position with numpy : 0.0017540454864501953 nb_pixel_total : 35735 time to create 1 rle with old method : 0.04126167297363281 length of segment : 325 time for calcul the mask position with numpy : 0.0009646415710449219 nb_pixel_total : 18768 time to create 1 rle with old method : 0.030470848083496094 length of segment : 132 time for calcul the mask position with numpy : 0.00034427642822265625 nb_pixel_total : 7913 time to create 1 rle with old method : 0.012940168380737305 length of segment : 119 time for calcul the mask position with numpy : 0.001501321792602539 nb_pixel_total : 27327 time to create 1 rle with old method : 0.03176593780517578 length of segment : 188 time for calcul the mask position with numpy : 0.0030546188354492188 nb_pixel_total : 65311 time to create 1 rle with old method : 0.07530021667480469 length of segment : 332 time for calcul the mask position with numpy : 0.001207113265991211 nb_pixel_total : 28819 time to create 1 rle with old method : 0.03255271911621094 length of segment : 328 time for calcul the mask position with numpy : 0.00524449348449707 nb_pixel_total : 165842 time to create 1 rle with new method : 0.00800013542175293 length of segment : 196 time for calcul the mask position with numpy : 0.0031824111938476562 nb_pixel_total : 53295 time to create 1 rle with old method : 0.06222796440124512 length of segment : 327 time for calcul the mask position with numpy : 0.000614166259765625 nb_pixel_total : 29031 time to create 1 rle with old method : 0.033795833587646484 length of segment : 159 time for calcul the mask position with numpy : 0.0008141994476318359 nb_pixel_total : 22861 time to create 1 rle with old method : 0.026686668395996094 length of segment : 128 time for calcul the mask position with numpy : 0.0028066635131835938 nb_pixel_total : 71598 time to create 1 rle with old method : 0.10360836982727051 length of segment : 307 time for calcul the mask position with numpy : 0.003529787063598633 nb_pixel_total : 88903 time to create 1 rle with old method : 0.10548138618469238 length of segment : 679 time for calcul the mask position with numpy : 0.0009069442749023438 nb_pixel_total : 11036 time to create 1 rle with old method : 0.014708280563354492 length of segment : 318 time for calcul the mask position with numpy : 0.0014150142669677734 nb_pixel_total : 34685 time to create 1 rle with old method : 0.04086875915527344 length of segment : 208 time for calcul the mask position with numpy : 0.0018091201782226562 nb_pixel_total : 29944 time to create 1 rle with old method : 0.037757158279418945 length of segment : 277 time for calcul the mask position with numpy : 0.004777431488037109 nb_pixel_total : 121638 time to create 1 rle with old method : 0.1410388946533203 length of segment : 484 time for calcul the mask position with numpy : 0.0035195350646972656 nb_pixel_total : 62566 time to create 1 rle with old method : 0.07410764694213867 length of segment : 668 time for calcul the mask position with numpy : 0.0063114166259765625 nb_pixel_total : 124950 time to create 1 rle with old method : 0.14171600341796875 length of segment : 667 time for calcul the mask position with numpy : 0.0032775402069091797 nb_pixel_total : 58027 time to create 1 rle with old method : 0.06658148765563965 length of segment : 333 time spent for convertir_results : 7.594292402267456 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 64 chid ids of type : 3594 Number RLEs to save : 17996 save missing photos in datou_result : time spend for datou_step_exec : 47.66279435157776 time spend to save output : 1.242055892944336 total time spend for step 1 : 48.904850244522095 step2:crop_condition Tue May 20 17:01:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 4 ! batch 1 Loaded 64 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 49 About to insert : list_path_to_insert length 49 new photo from crops ! About to upload 49 photos upload in portfolio : 3736932 init cache_photo without model_param we have 49 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747753293_3148537 we have uploaded 49 photos in the portfolio 3736932 time of upload the photos Elapsed time : 15.658010005950928 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747753311_3148537 we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.4850175380706787 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3736932 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747753319_3148537 we have uploaded 8 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.580576181411743 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747753323_3148537 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8273193836212158 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1359813976, 1359813973, 1359813727, 1359813723] Looping around the photos to save general results len do output : 64 /1359829308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1359829463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813976', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813973', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813727', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813723', None, None, None, None, None, '2909932') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 196 time used for this insertion : 0.024255990982055664 save_final save missing photos in datou_result : time spend for datou_step_exec : 43.08272194862366 time spend to save output : 0.027144432067871094 total time spend for step 2 : 43.10986638069153 step3:rle_unique_nms_with_priority Tue May 20 17:02:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 64 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 4.314767599105835 time for calcul the mask position with numpy : 0.409090518951416 nb_pixel_total : 7634309 time to create 1 rle with new method : 0.4571197032928467 time for calcul the mask position with numpy : 0.03393864631652832 nb_pixel_total : 100091 time to create 1 rle with old method : 0.11244392395019531 time for calcul the mask position with numpy : 0.02819991111755371 nb_pixel_total : 34959 time to create 1 rle with old method : 0.04157686233520508 time for calcul the mask position with numpy : 0.030730009078979492 nb_pixel_total : 55532 time to create 1 rle with old method : 0.07080745697021484 time for calcul the mask position with numpy : 0.03430771827697754 nb_pixel_total : 72334 time to create 1 rle with old method : 0.09089159965515137 time for calcul the mask position with numpy : 0.026395320892333984 nb_pixel_total : 8037 time to create 1 rle with old method : 0.009191036224365234 time for calcul the mask position with numpy : 0.029961109161376953 nb_pixel_total : 389138 time to create 1 rle with new method : 0.47506046295166016 create new chi : 1.9137814044952393 time to delete rle : 0.029604196548461914 batch 1 Loaded 13 chid ids of type : 3594 ++++++++Number RLEs to save : 5970 TO DO : save crop sub photo not yet done ! save time : 0.40030694007873535 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 7.038518667221069 time for calcul the mask position with numpy : 0.7064042091369629 nb_pixel_total : 7982167 time to create 1 rle with new method : 1.0531563758850098 time for calcul the mask position with numpy : 0.03991222381591797 nb_pixel_total : 32633 time to create 1 rle with old method : 0.03799629211425781 time for calcul the mask position with numpy : 0.04284334182739258 nb_pixel_total : 31101 time to create 1 rle with old method : 0.0498967170715332 time for calcul the mask position with numpy : 0.05029702186584473 nb_pixel_total : 40785 time to create 1 rle with old method : 0.047106027603149414 time for calcul the mask position with numpy : 0.04176759719848633 nb_pixel_total : 7642 time to create 1 rle with old method : 0.008469581604003906 time for calcul the mask position with numpy : 0.04073333740234375 nb_pixel_total : 65470 time to create 1 rle with old method : 0.07411026954650879 time for calcul the mask position with numpy : 0.0416562557220459 nb_pixel_total : 27154 time to create 1 rle with old method : 0.030335187911987305 time for calcul the mask position with numpy : 0.04370570182800293 nb_pixel_total : 11380 time to create 1 rle with old method : 0.01825571060180664 time for calcul the mask position with numpy : 0.04931759834289551 nb_pixel_total : 6449 time to create 1 rle with old method : 0.010291099548339844 time for calcul the mask position with numpy : 0.036199092864990234 nb_pixel_total : 33080 time to create 1 rle with old method : 0.04212689399719238 time for calcul the mask position with numpy : 0.02645707130432129 nb_pixel_total : 19892 time to create 1 rle with old method : 0.023252487182617188 time for calcul the mask position with numpy : 0.026587724685668945 nb_pixel_total : 36647 time to create 1 rle with old method : 0.043662309646606445 create new chi : 2.6292572021484375 time to delete rle : 0.0011990070343017578 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 6630 TO DO : save crop sub photo not yet done ! save time : 0.443373441696167 nb_obj : 22 nb_hashtags : 4 time to prepare the origin masks : 11.090695858001709 time for calcul the mask position with numpy : 0.34429359436035156 nb_pixel_total : 6945141 time to create 1 rle with new method : 0.885704755783081 time for calcul the mask position with numpy : 0.04144763946533203 nb_pixel_total : 51405 time to create 1 rle with old method : 0.058109283447265625 time for calcul the mask position with numpy : 0.04640769958496094 nb_pixel_total : 18503 time to create 1 rle with old method : 0.03022909164428711 time for calcul the mask position with numpy : 0.04822683334350586 nb_pixel_total : 10112 time to create 1 rle with old method : 0.011832714080810547 time for calcul the mask position with numpy : 0.04324531555175781 nb_pixel_total : 36711 time to create 1 rle with old method : 0.04027438163757324 time for calcul the mask position with numpy : 0.04049324989318848 nb_pixel_total : 110388 time to create 1 rle with old method : 0.1344594955444336 time for calcul the mask position with numpy : 0.043328285217285156 nb_pixel_total : 9913 time to create 1 rle with old method : 0.011263132095336914 time for calcul the mask position with numpy : 0.040990352630615234 nb_pixel_total : 37802 time to create 1 rle with old method : 0.04291057586669922 time for calcul the mask position with numpy : 0.040508270263671875 nb_pixel_total : 31816 time to create 1 rle with old method : 0.03625750541687012 time for calcul the mask position with numpy : 0.04109382629394531 nb_pixel_total : 73817 time to create 1 rle with old method : 0.0876471996307373 time for calcul the mask position with numpy : 0.041327714920043945 nb_pixel_total : 30626 time to create 1 rle with old method : 0.03445863723754883 time for calcul the mask position with numpy : 0.051911354064941406 nb_pixel_total : 31462 time to create 1 rle with old method : 0.05820584297180176 time for calcul the mask position with numpy : 0.04814457893371582 nb_pixel_total : 54040 time to create 1 rle with old method : 0.06536316871643066 time for calcul the mask position with numpy : 0.044606924057006836 nb_pixel_total : 33483 time to create 1 rle with old method : 0.03926801681518555 time for calcul the mask position with numpy : 0.0408320426940918 nb_pixel_total : 16711 time to create 1 rle with old method : 0.025439977645874023 time for calcul the mask position with numpy : 0.05595803260803223 nb_pixel_total : 283643 time to create 1 rle with new method : 1.0532405376434326 time for calcul the mask position with numpy : 0.04138827323913574 nb_pixel_total : 28269 time to create 1 rle with old method : 0.03468513488769531 time for calcul the mask position with numpy : 0.04723215103149414 nb_pixel_total : 75450 time to create 1 rle with old method : 0.0894172191619873 time for calcul the mask position with numpy : 0.04778456687927246 nb_pixel_total : 87892 time to create 1 rle with old method : 0.09916043281555176 time for calcul the mask position with numpy : 0.04514718055725098 nb_pixel_total : 81754 time to create 1 rle with old method : 0.09426498413085938 time for calcul the mask position with numpy : 0.04269576072692871 nb_pixel_total : 21334 time to create 1 rle with old method : 0.02371525764465332 time for calcul the mask position with numpy : 0.0404505729675293 nb_pixel_total : 133944 time to create 1 rle with old method : 0.14839506149291992 time for calcul the mask position with numpy : 0.03390932083129883 nb_pixel_total : 90184 time to create 1 rle with old method : 0.10381865501403809 create new chi : 4.595094442367554 time to delete rle : 0.002254009246826172 batch 1 Loaded 45 chid ids of type : 3594 ++++++++++++++++++++++++++Number RLEs to save : 13685 TO DO : save crop sub photo not yet done ! save time : 0.8520426750183105 nb_obj : 25 nb_hashtags : 3 time to prepare the origin masks : 11.266879796981812 time for calcul the mask position with numpy : 0.6753091812133789 nb_pixel_total : 6886406 time to create 1 rle with new method : 1.168757438659668 time for calcul the mask position with numpy : 0.04122018814086914 nb_pixel_total : 58027 time to create 1 rle with old method : 0.06488394737243652 time for calcul the mask position with numpy : 0.0409238338470459 nb_pixel_total : 124938 time to create 1 rle with old method : 0.1370103359222412 time for calcul the mask position with numpy : 0.038630008697509766 nb_pixel_total : 62566 time to create 1 rle with old method : 0.0682680606842041 time for calcul the mask position with numpy : 0.03817009925842285 nb_pixel_total : 121638 time to create 1 rle with old method : 0.13324689865112305 time for calcul the mask position with numpy : 0.035138845443725586 nb_pixel_total : 29944 time to create 1 rle with old method : 0.032755374908447266 time for calcul the mask position with numpy : 0.033120155334472656 nb_pixel_total : 34685 time to create 1 rle with old method : 0.0389401912689209 time for calcul the mask position with numpy : 0.02710747718811035 nb_pixel_total : 11036 time to create 1 rle with old method : 0.012146949768066406 time for calcul the mask position with numpy : 0.03553962707519531 nb_pixel_total : 88903 time to create 1 rle with old method : 0.09919333457946777 time for calcul the mask position with numpy : 0.028060197830200195 nb_pixel_total : 71598 time to create 1 rle with old method : 0.08048224449157715 time for calcul the mask position with numpy : 0.0319368839263916 nb_pixel_total : 22861 time to create 1 rle with old method : 0.026341915130615234 time for calcul the mask position with numpy : 0.026825428009033203 nb_pixel_total : 29031 time to create 1 rle with old method : 0.03185129165649414 time for calcul the mask position with numpy : 0.02781510353088379 nb_pixel_total : 53295 time to create 1 rle with old method : 0.05887651443481445 time for calcul the mask position with numpy : 0.027013778686523438 nb_pixel_total : 165842 time to create 1 rle with new method : 0.9875783920288086 time for calcul the mask position with numpy : 0.02523183822631836 nb_pixel_total : 28819 time to create 1 rle with old method : 0.03524589538574219 time for calcul the mask position with numpy : 0.027312755584716797 nb_pixel_total : 65311 time to create 1 rle with old method : 0.07376360893249512 time for calcul the mask position with numpy : 0.026596784591674805 nb_pixel_total : 27327 time to create 1 rle with old method : 0.030962705612182617 time for calcul the mask position with numpy : 0.02540445327758789 nb_pixel_total : 7913 time to create 1 rle with old method : 0.008951902389526367 time for calcul the mask position with numpy : 0.02547287940979004 nb_pixel_total : 18768 time to create 1 rle with old method : 0.020698070526123047 time for calcul the mask position with numpy : 0.025537490844726562 nb_pixel_total : 35735 time to create 1 rle with old method : 0.03955841064453125 time for calcul the mask position with numpy : 0.026285886764526367 nb_pixel_total : 65828 time to create 1 rle with old method : 0.07480883598327637 time for calcul the mask position with numpy : 0.02637314796447754 nb_pixel_total : 32069 time to create 1 rle with old method : 0.03580069541931152 time for calcul the mask position with numpy : 0.02545952796936035 nb_pixel_total : 47893 time to create 1 rle with old method : 0.053731679916381836 time for calcul the mask position with numpy : 0.025228261947631836 nb_pixel_total : 23546 time to create 1 rle with old method : 0.0301511287689209 time for calcul the mask position with numpy : 0.0304872989654541 nb_pixel_total : 122217 time to create 1 rle with old method : 0.20554876327514648 time for calcul the mask position with numpy : 0.0295257568359375 nb_pixel_total : 58204 time to create 1 rle with old method : 0.06599593162536621 create new chi : 5.116806983947754 time to delete rle : 0.0021681785583496094 batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++Number RLEs to save : 18307 TO DO : save crop sub photo not yet done ! save time : 1.1819579601287842 map_output_result : {1359813976: (0.0, 'Should be the crop_list due to order', 0), 1359813973: (0.0, 'Should be the crop_list due to order', 0), 1359813727: (0.0, 'Should be the crop_list due to order', 0), 1359813723: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1359813976, 1359813973, 1359813727, 1359813723] Looping around the photos to save general results len do output : 4 /1359813976.Didn't retrieve data . /1359813973.Didn't retrieve data . /1359813727.Didn't retrieve data . /1359813723.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813976', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813973', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813727', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813723', None, None, None, None, None, '2909932') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.018644094467163086 save_final save missing photos in datou_result : time spend for datou_step_exec : 52.209283113479614 time spend to save output : 0.019145488739013672 total time spend for step 3 : 52.22842860221863 step4:ventilate_hashtags_in_portfolio Tue May 20 17:02:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23171910 get user id for portfolio 23171910 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23171910 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','mal_croppe','background','pet_fonce','pet_clair','papier','autre','pehd','carton','metal','environnement')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23171910 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','mal_croppe','background','pet_fonce','pet_clair','papier','autre','pehd','carton','metal','environnement')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23172950,23172951,23172952,23172953,23172954,23172955,23172956,23172957,23172958,23172959,23172960?tags=carton,pehd,metal,pet_fonce,flou,background,papier,pet_clair,environnement,autre,mal_croppe Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1359813976, 1359813973, 1359813727, 1359813723] Looping around the photos to save general results len do output : 1 /23171910. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813976', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813973', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813727', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813723', None, None, None, None, None, '2909932') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.015584230422973633 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.5002646446228027 time spend to save output : 0.01912713050842285 total time spend for step 4 : 0.5193917751312256 step5:final Tue May 20 17:02:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1359813976: ('0.11759584219333181',), 1359813973: ('0.11759584219333181',), 1359813727: ('0.11759584219333181',), 1359813723: ('0.11759584219333181',)} new output for save of step final : {1359813976: ('0.11759584219333181',), 1359813973: ('0.11759584219333181',), 1359813727: ('0.11759584219333181',), 1359813723: ('0.11759584219333181',)} [1359813976, 1359813973, 1359813727, 1359813723] Looping around the photos to save general results len do output : 4 /1359813976.Didn't retrieve data . /1359813973.Didn't retrieve data . /1359813727.Didn't retrieve data . /1359813723.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813976', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813973', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813727', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813723', None, None, None, None, None, '2909932') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.020259857177734375 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12735533714294434 time spend to save output : 0.020951271057128906 total time spend for step 5 : 0.14830660820007324 step6:blur_detection Tue May 20 17:02:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b.jpg resize: (2160, 3840) 1359813976 -3.8265002085977535 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5.jpg resize: (2160, 3840) 1359813973 -5.673919091786592 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15.jpg resize: (2160, 3840) 1359813727 -5.497949514756715 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff.jpg resize: (2160, 3840) 1359813723 -5.954928279018027 treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426438_0.png resize: (98, 119) 1359829308 -4.296760343632532 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426443_0.png resize: (249, 230) 1359829309 -3.800958750031823 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426444_0.png resize: (160, 167) 1359829310 -3.1560902540765636 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426445_0.png resize: (203, 253) 1359829311 -0.9498729291506132 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426447_0.png resize: (97, 150) 1359829312 1.627357821740108 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426448_0.png resize: (225, 289) 1359829313 -4.4022803407685895 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426449_0.png resize: (327, 254) 1359829314 -2.775316278598029 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426450_0.png resize: (74, 173) 1359829315 -2.9697045814431986 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426451_0.png resize: (159, 388) 1359829316 -3.7434981769310074 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426452_0.png resize: (300, 160) 1359829317 -4.518069149811428 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426453_0.png resize: (236, 313) 1359829318 -2.621996897260624 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426455_0.png resize: (314, 672) 1359829319 -3.130051734651706 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426456_0.png resize: (114, 253) 1359829320 -2.549037575909744 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426457_0.png resize: (321, 406) 1359829321 -4.439001395915161 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426458_0.png resize: (349, 420) 1359829322 -1.3247265473723195 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426460_0.png resize: (169, 242) 1359829323 -4.011715403967513 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426463_0.png resize: (271, 227) 1359829324 -2.7263186774863923 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426464_0.png resize: (393, 252) 1359829325 -2.0753699720989256 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426465_0.png resize: (162, 245) 1359829326 -3.4222112537702056 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426466_0.png resize: (205, 229) 1359829327 -2.0320054503793297 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426467_0.png resize: (315, 360) 1359829328 -3.9816358075642366 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426468_0.png resize: (168, 277) 1359829330 -4.024755672084954 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426470_0.png resize: (92, 155) 1359829331 -3.226025717570372 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426471_0.png resize: (427, 495) 1359829332 -3.8688312183655293 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426472_0.png resize: (302, 187) 1359829333 -3.4366624507241563 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426473_0.png resize: (141, 98) 1359829334 -3.212811010794673 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426474_0.png resize: (150, 178) 1359829335 -2.463390795899545 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426475_0.png resize: (210, 317) 1359829336 -1.8439099213737185 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426476_0.png resize: (246, 353) 1359829337 -4.146016267744541 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426477_0.png resize: (348, 508) 1359829338 -4.003467013350003 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426478_0.png resize: (202, 168) 1359829339 -2.4569070226033776 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426480_0.png resize: (176, 277) 1359829340 -3.043419507550513 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426481_0.png resize: (329, 391) 1359829341 -0.02292523472941773 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426482_0.png resize: (295, 180) 1359829342 -4.02625007231792 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426483_0.png resize: (130, 199) 1359829343 -3.7175104184430348 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426484_0.png resize: (117, 100) 1359829344 -3.7455309729302915 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426486_0.png resize: (322, 362) 1359829345 -3.246301331333795 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426487_0.png resize: (325, 111) 1359829346 -2.4723888215496355 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426489_0.png resize: (243, 430) 1359829347 -1.4370851093297896 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426490_0.png resize: (159, 242) 1359829348 -4.771896648305579 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426492_0.png resize: (307, 322) 1359829350 -2.6673010865574205 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426493_0.png resize: (377, 473) 1359829351 -0.9371320044697654 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426494_0.png resize: (241, 99) 1359829352 -4.035709826843539 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426495_0.png resize: (182, 239) 1359829353 -0.5733888193660568 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426496_0.png resize: (196, 241) 1359829354 -2.494673794073837 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426497_0.png resize: (335, 507) 1359829355 -3.3691474841026734 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426498_0.png resize: (543, 205) 1359829356 -4.504142577690121 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426499_0.png resize: (521, 573) 1359829357 -4.102506522153774 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426500_0.png resize: (273, 314) 1359829358 -1.4784609981193975 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426454_0.png resize: (301, 507) 1359829373 -3.1679045101107657 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426462_0.png resize: (219, 105) 1359829374 -3.6769152746548617 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426479_0.png resize: (362, 245) 1359829376 -4.28630824534677 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426485_0.png resize: (183, 250) 1359829377 -3.6504733067626307 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426491_0.png resize: (127, 222) 1359829378 -3.8330934134352552 treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426437_0.png resize: (396, 1412) 1359829437 -3.0005665313647336 treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426439_0.png resize: (345, 263) 1359829438 -3.8794883669618097 treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426440_0.png resize: (284, 323) 1359829440 -4.208788489928944 treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426441_0.png resize: (256, 220) 1359829441 -3.925021958334347 treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426442_0.png resize: (385, 472) 1359829442 -3.896489696971346 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426459_0.png resize: (330, 311) 1359829443 -4.028094362502379 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426461_0.png resize: (371, 1467) 1359829445 -3.8316078322918106 treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426488_0.png resize: (191, 1111) 1359829446 -4.430754630417327 treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426446_0.png resize: (85, 96) 1359829461 -3.6149234270143826 treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426469_0.png resize: (162, 271) 1359829463 -2.054261713589339 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 68 time used for this insertion : 0.01593947410583496 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 68 time used for this insertion : 0.019666433334350586 save missing photos in datou_result : time spend for datou_step_exec : 18.4252290725708 time spend to save output : 0.04093623161315918 total time spend for step 6 : 18.46616530418396 step7:brightness Tue May 20 17:03:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b.jpg treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5.jpg treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15.jpg treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff.jpg treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426438_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426443_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426444_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426445_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426447_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426448_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426449_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426450_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426451_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426452_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426453_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426455_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426456_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426457_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426458_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426460_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426463_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426464_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426465_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426466_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426467_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426468_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426470_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426471_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426472_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426473_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426474_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426475_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426476_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426477_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426478_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426480_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426481_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426482_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426483_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426484_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426486_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426487_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426489_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426490_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426492_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426493_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426494_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426495_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426496_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426497_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426498_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426499_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426500_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426454_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426462_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426479_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426485_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426491_0.png treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426437_0.png treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426439_0.png treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426440_0.png treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426441_0.png treat image : temp/1747753230_3148537_1359813976_f290fcd7247c29104997a4776b6af65b_rle_crop_3806426442_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426459_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426461_0.png treat image : temp/1747753230_3148537_1359813723_397e887ffc1daf2aff509bc7be883aff_rle_crop_3806426488_0.png treat image : temp/1747753230_3148537_1359813973_ae07483f7039e9dd4dadee928df022c5_rle_crop_3806426446_0.png treat image : temp/1747753230_3148537_1359813727_7fe5cd1c8fe27647f9863eb9f505cc15_rle_crop_3806426469_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 68 time used for this insertion : 0.017095327377319336 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 68 time used for this insertion : 0.014664649963378906 save missing photos in datou_result : time spend for datou_step_exec : 4.803398132324219 time spend to save output : 0.03673410415649414 total time spend for step 7 : 4.840132236480713 step8:velours_tree Tue May 20 17:03:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.23476099967956543 time spend to save output : 4.57763671875e-05 total time spend for step 8 : 0.23480677604675293 step9:send_mail_cod Tue May 20 17:03:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23171910_20-05-2025_17_03_19.pdf 23172950 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231729501747753399 23172951 imagette231729511747753402 23172952 change filename to text .change filename to text .change filename to text -- Missing : http://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/20/e5d185fe46b307b4fbff185f829b063f_92013975.png retrying : [Errno 104] Connection reset by peer change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231729521747753402 23172953 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231729531747753403 23172954 imagette231729541747753404 23172955 imagette231729551747753404 23172956 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231729561747753404 23172957 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text -- Missing : http://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/20/29a789273a8b45112b71431badbfaa4d_71202370.png retrying : [Errno 104] Connection reset by peer change filename to text .change filename to text .imagette231729571747753406 23172959 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette231729591747753408 23172960 imagette231729601747753409 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23171910 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23172950,23172951,23172952,23172953,23172954,23172955,23172956,23172957,23172958,23172959,23172960?tags=carton,pehd,metal,pet_fonce,flou,background,papier,pet_clair,environnement,autre,mal_croppe args[1359813976] : ((1359813976, -3.8265002085977535, 492609224), (1359813976, -0.010433431598845569, 2107752395), '0.11759584219333181') We are sending mail with results at report@fotonower.com args[1359813973] : ((1359813973, -5.673919091786592, 492609224), (1359813973, 0.34660001558215414, 2107752395), '0.11759584219333181') We are sending mail with results at report@fotonower.com args[1359813727] : ((1359813727, -5.497949514756715, 492609224), (1359813727, 0.44179401918678046, 2107752395), '0.11759584219333181') We are sending mail with results at report@fotonower.com args[1359813723] : ((1359813723, -5.954928279018027, 492609224), (1359813723, 1.066600594210104, 2107752395), '0.11759584219333181') We are sending mail with results at report@fotonower.com refus_total : 0.11759584219333181 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23171910 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 error cannot identify image file 'imagette231729521747753402/1747750230_2955930_1359816042_3afb3b1feffde5b0187b8023c2ddcaf4_rle_crop_3806404126_0.png.png' error cannot identify image file 'imagette231729571747753406/1747750230_2955930_1359816081_9d483db809806307c3a7c644cb43b961_rle_crop_3806404090_0.png.png' start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23171910_20-05-2025_17_03_19.pdf results_Auto_P23171910_20-05-2025_17_03_19.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23171910_20-05-2025_17_03_19.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23171910','results_Auto_P23171910_20-05-2025_17_03_19.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23171910_20-05-2025_17_03_19.pdf','pdf','','1.01','0.11759584219333181') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23171910

https://www.fotonower.com/image?json=false&list_photos_id=1359813976
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359813973
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359813727
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1359813723
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 11.76%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/23172950?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23172952?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23172953?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23172956?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23172957?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23172959?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23171910_20-05-2025_17_03_19.pdf.

Lien vers velours :https://www.fotonower.com/velours/23172950,23172951,23172952,23172953,23172954,23172955,23172956,23172957,23172958,23172959,23172960?tags=carton,pehd,metal,pet_fonce,flou,background,papier,pet_clair,environnement,autre,mal_croppe.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 20 May 2025 15:03:32 GMT Content-Length: 0 Connection: close X-Message-Id: woz70hqPQfCZZ0bWuyXeTA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1359813976, 1359813973, 1359813727, 1359813723] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813976', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813973', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813727', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813723', None, None, None, None, None, '2909932') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.014740943908691406 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.32586407661438 time spend to save output : 0.015052556991577148 total time spend for step 9 : 12.340916633605957 step10:split_time_score Tue May 20 17:03:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 18), ('15', 26)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20052025 23171910 Nombre de photos uploadées : 44 / 23040 (0%) 20052025 23171910 Nombre de photos taguées (types de déchets): 0 / 44 (0%) 20052025 23171910 Nombre de photos taguées (volume) : 0 / 44 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.001957416534423828 elapsed_time : insert_dashboard_record_day_entry 0.023888349533081055 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.09201400421027917 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23157181_20-05-2025_08_46_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23157181 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23157181 AND mptpi.`type`=3594 To do Qualite : 0.10180426772060878 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23157188_20-05-2025_08_35_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23157188 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23157188 AND mptpi.`type`=3594 To do Qualite : 0.09871742380401231 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23157192_20-05-2025_08_28_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23157192 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23157192 AND mptpi.`type`=3594 To do Qualite : 0.10408518822159352 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23161557_20-05-2025_10_30_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23161557 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23161557 AND mptpi.`type`=3594 To do Qualite : 0.1308608699845679 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23164737_20-05-2025_11_54_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23164737 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23164737 AND mptpi.`type`=3594 To do Qualite : 0.1064864810896508 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23164744_20-05-2025_12_03_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23164744 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23164744 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170102 order by id desc limit 1 Qualite : 0.11795572916666669 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170104_20-05-2025_15_34_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170104 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170104 AND mptpi.`type`=3594 To do Qualite : 0.02726871178050528 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23166958_20-05-2025_13_30_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23166958 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23166958 AND mptpi.`type`=3726 To do Qualite : 0.09543954354745371 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23166961_20-05-2025_13_20_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23166961 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23166961 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170110 order by id desc limit 1 Qualite : 0.10659472484016758 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23170112_20-05-2025_15_29_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23170112 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23170112 AND mptpi.`type`=3594 To do Qualite : 0.11812746691001888 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23171910_20-05-2025_17_03_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23171910 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23171910 AND mptpi.`type`=3594 To do Qualite : 0.19817297078617974 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23172899_20-05-2025_16_57_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23172899 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23172899 AND mptpi.`type`=3594 To do Qualite : 0.20049805783529734 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23172907_20-05-2025_16_49_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23172907 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23172907 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20052025': {'nb_upload': 44, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359813976, 1359813973, 1359813727, 1359813723] Looping around the photos to save general results len do output : 1 /23171910Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813976', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813973', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813727', None, None, None, None, None, '2909932') ('3318', None, None, None, None, None, None, None, '2909932') ('3318', None, '1359813723', None, None, None, None, None, '2909932') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.01474618911743164 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.9876606464385986 time spend to save output : 0.014940738677978516 total time spend for step 10 : 3.002601385116577 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 4 set_done_treatment 75.81user 74.71system 3:09.34elapsed 79%CPU (0avgtext+0avgdata 4036768maxresident)k 675408inputs+105600outputs (9326major+7332173minor)pagefaults 0swaps