python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3827973 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3681349'] with mtr_portfolio_ids : ['26666422'] and first list_photo_ids : [] new path : /proc/3827973/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 18 ; length of list_pids : 18 ; length of list_args : 18 time to download the photos : 2.5042335987091064 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Sep 9 00:30:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10597 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-09 00:30:35.136766: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-09 00:30:35.160477: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-09 00:30:35.162156: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f9cb0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-09 00:30:35.162202: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-09 00:30:35.165551: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-09 00:30:35.283548: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xb7becf0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-09 00:30:35.283619: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-09 00:30:35.285005: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-09 00:30:35.285518: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-09 00:30:35.289157: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-09 00:30:35.292466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-09 00:30:35.293014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-09 00:30:35.296913: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-09 00:30:35.298218: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-09 00:30:35.303231: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-09 00:30:35.304805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-09 00:30:35.304892: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-09 00:30:35.305696: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-09 00:30:35.305713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-09 00:30:35.305721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-09 00:30:35.307061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9672 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-09 00:30:35.591061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-09 00:30:35.591192: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-09 00:30:35.591220: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-09 00:30:35.591245: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-09 00:30:35.591270: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-09 00:30:35.591303: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-09 00:30:35.591333: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-09 00:30:35.591365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-09 00:30:35.593246: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-09 00:30:35.594959: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-09 00:30:35.595062: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-09 00:30:35.595104: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-09 00:30:35.595140: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-09 00:30:35.595179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-09 00:30:35.595216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-09 00:30:35.595250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-09 00:30:35.595285: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-09 00:30:35.597198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-09 00:30:35.597260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-09 00:30:35.597276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-09 00:30:35.597288: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-09 00:30:35.599237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9672 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-09 00:30:44.760785: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-09 00:30:44.962137: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 18 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 23.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 3828746 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 335 tf kernel not reseted sub process len(results) : 18 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 18 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5624 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.011568546295166016 nb_pixel_total : 565040 time to create 1 rle with new method : 0.04522848129272461 length of segment : 1330 time for calcul the mask position with numpy : 0.00019121170043945312 nb_pixel_total : 5396 time to create 1 rle with old method : 0.006386995315551758 length of segment : 118 time for calcul the mask position with numpy : 0.00011396408081054688 nb_pixel_total : 3352 time to create 1 rle with old method : 0.0039789676666259766 length of segment : 73 time for calcul the mask position with numpy : 0.010434389114379883 nb_pixel_total : 656866 time to create 1 rle with new method : 0.04117894172668457 length of segment : 1961 time for calcul the mask position with numpy : 0.00039124488830566406 nb_pixel_total : 23432 time to create 1 rle with old method : 0.024608135223388672 length of segment : 151 time for calcul the mask position with numpy : 0.0013904571533203125 nb_pixel_total : 101260 time to create 1 rle with old method : 0.11223793029785156 length of segment : 481 time for calcul the mask position with numpy : 0.00018668174743652344 nb_pixel_total : 10106 time to create 1 rle with old method : 0.011888980865478516 length of segment : 126 time for calcul the mask position with numpy : 9.131431579589844e-05 nb_pixel_total : 3792 time to create 1 rle with old method : 0.004680156707763672 length of segment : 44 time for calcul the mask position with numpy : 0.00026535987854003906 nb_pixel_total : 17277 time to create 1 rle with old method : 0.01941657066345215 length of segment : 181 time for calcul the mask position with numpy : 0.00020766258239746094 nb_pixel_total : 14262 time to create 1 rle with old method : 0.016889333724975586 length of segment : 194 time for calcul the mask position with numpy : 0.0001499652862548828 nb_pixel_total : 8519 time to create 1 rle with old method : 0.009508371353149414 length of segment : 147 time for calcul the mask position with numpy : 0.00011324882507324219 nb_pixel_total : 4612 time to create 1 rle with old method : 0.0052492618560791016 length of segment : 110 time for calcul the mask position with numpy : 8.654594421386719e-05 nb_pixel_total : 3946 time to create 1 rle with old method : 0.0045626163482666016 length of segment : 114 time for calcul the mask position with numpy : 0.0015184879302978516 nb_pixel_total : 107675 time to create 1 rle with old method : 0.12180161476135254 length of segment : 516 time for calcul the mask position with numpy : 0.0004684925079345703 nb_pixel_total : 24468 time to create 1 rle with old method : 0.028060197830200195 length of segment : 261 time for calcul the mask position with numpy : 0.0007295608520507812 nb_pixel_total : 44120 time to create 1 rle with old method : 0.050315141677856445 length of segment : 159 time for calcul the mask position with numpy : 0.0013363361358642578 nb_pixel_total : 87563 time to create 1 rle with old method : 0.09408926963806152 length of segment : 487 time for calcul the mask position with numpy : 9.226799011230469e-05 nb_pixel_total : 2748 time to create 1 rle with old method : 0.002998828887939453 length of segment : 74 time for calcul the mask position with numpy : 0.0016498565673828125 nb_pixel_total : 115207 time to create 1 rle with old method : 0.13128328323364258 length of segment : 536 time for calcul the mask position with numpy : 0.0020771026611328125 nb_pixel_total : 90287 time to create 1 rle with old method : 0.10145187377929688 length of segment : 565 time for calcul the mask position with numpy : 0.00042557716369628906 nb_pixel_total : 12922 time to create 1 rle with old method : 0.01431894302368164 length of segment : 144 time for calcul the mask position with numpy : 0.0022182464599609375 nb_pixel_total : 109490 time to create 1 rle with old method : 0.11451435089111328 length of segment : 534 time for calcul the mask position with numpy : 0.0002455711364746094 nb_pixel_total : 6351 time to create 1 rle with old method : 0.007081031799316406 length of segment : 99 time for calcul the mask position with numpy : 0.0006399154663085938 nb_pixel_total : 19674 time to create 1 rle with old method : 0.02131032943725586 length of segment : 189 time for calcul the mask position with numpy : 0.032608985900878906 nb_pixel_total : 814491 time to create 1 rle with new method : 0.21567749977111816 length of segment : 1354 time spent for convertir_results : 3.649491548538208 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 25 chid ids of type : 3594 Number RLEs to save : 9948 save missing photos in datou_result : time spend for datou_step_exec : 29.687268257141113 time spend to save output : 0.6744406223297119 total time spend for step 1 : 30.361708879470825 step2:crop_condition Tue Sep 9 00:31:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 18 ! batch 1 Loaded 25 chid ids of type : 3594 ++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757370663_3827973 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.5637381076812744 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757370668_3827973 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.2391595840454102 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 3736932 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757370678_3827973 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 16 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.659722805023193 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757370683_3827973 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5876240730285645 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1382616575, 1382616560, 1382616547, 1382616540, 1382616530, 1382616499, 1382616497, 1382616494, 1382616491, 1382616489, 1382616486, 1382616323, 1382616295, 1382616256, 1382616228, 1382616198, 1382616171, 1382615800] Looping around the photos to save general results len do output : 25 /1382701183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1382701214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616575', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616560', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616547', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616540', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616530', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616499', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616497', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616494', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616491', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616489', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616486', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616323', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616295', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616256', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616228', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616198', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616171', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382615800', None, None, None, None, None, '3681349') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 93 time used for this insertion : 0.025484085083007812 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.15380334854126 time spend to save output : 0.02643275260925293 total time spend for step 2 : 21.180236101150513 step3:rle_unique_nms_with_priority Tue Sep 9 00:31:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 25 chid ids of type : 3594 ++++++++++++++++++++++++++++++++nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.40591931343078613 time for calcul the mask position with numpy : 0.22805523872375488 nb_pixel_total : 1503164 time to create 1 rle with new method : 0.09929323196411133 time for calcul the mask position with numpy : 0.00788736343383789 nb_pixel_total : 5396 time to create 1 rle with old method : 0.006166934967041016 time for calcul the mask position with numpy : 0.017550230026245117 nb_pixel_total : 565040 time to create 1 rle with new method : 0.3168752193450928 create new chi : 0.698113203048706 time to delete rle : 0.028268814086914062 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 3976 TO DO : save crop sub photo not yet done ! save time : 0.49216365814208984 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.19918322563171387 time for calcul the mask position with numpy : 0.6152522563934326 nb_pixel_total : 1416734 time to create 1 rle with new method : 0.12944746017456055 time for calcul the mask position with numpy : 0.016937732696533203 nb_pixel_total : 653514 time to create 1 rle with new method : 0.4764218330383301 time for calcul the mask position with numpy : 0.008767127990722656 nb_pixel_total : 3352 time to create 1 rle with old method : 0.003758668899536133 create new chi : 1.2713642120361328 time to delete rle : 0.000934600830078125 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 5148 TO DO : save crop sub photo not yet done ! save time : 0.6510472297668457 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.06022357940673828 time for calcul the mask position with numpy : 0.048029422760009766 nb_pixel_total : 2050168 time to create 1 rle with new method : 0.26308250427246094 time for calcul the mask position with numpy : 0.008692502975463867 nb_pixel_total : 23432 time to create 1 rle with old method : 0.02622699737548828 create new chi : 0.3601338863372803 time to delete rle : 0.00043582916259765625 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1382 TO DO : save crop sub photo not yet done ! save time : 0.12190365791320801 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.09371399879455566 time for calcul the mask position with numpy : 0.12722373008728027 nb_pixel_total : 1972340 time to create 1 rle with new method : 0.10471749305725098 time for calcul the mask position with numpy : 0.008697986602783203 nb_pixel_total : 101260 time to create 1 rle with old method : 0.11288094520568848 create new chi : 0.3675110340118408 time to delete rle : 0.0005095005035400391 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2042 TO DO : save crop sub photo not yet done ! save time : 0.3222076892852783 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.15329599380493164 time for calcul the mask position with numpy : 0.08140730857849121 nb_pixel_total : 2059702 time to create 1 rle with new method : 0.49007582664489746 time for calcul the mask position with numpy : 0.012793540954589844 nb_pixel_total : 3792 time to create 1 rle with old method : 0.00432896614074707 time for calcul the mask position with numpy : 0.00796365737915039 nb_pixel_total : 10106 time to create 1 rle with old method : 0.011523723602294922 create new chi : 0.6213507652282715 time to delete rle : 0.0006275177001953125 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 1420 TO DO : save crop sub photo not yet done ! save time : 0.13608026504516602 No data in photo_id : 1382616499 No data in photo_id : 1382616497 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.05423378944396973 time for calcul the mask position with numpy : 0.19045495986938477 nb_pixel_total : 2056323 time to create 1 rle with new method : 0.10579705238342285 time for calcul the mask position with numpy : 0.0075266361236572266 nb_pixel_total : 17277 time to create 1 rle with old method : 0.01936173439025879 create new chi : 0.33461594581604004 time to delete rle : 0.0004191398620605469 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1442 TO DO : save crop sub photo not yet done ! save time : 0.12168717384338379 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.07635617256164551 time for calcul the mask position with numpy : 0.05899357795715332 nb_pixel_total : 2059338 time to create 1 rle with new method : 0.3740715980529785 time for calcul the mask position with numpy : 0.007958650588989258 nb_pixel_total : 14262 time to create 1 rle with old method : 0.01678752899169922 create new chi : 0.4705827236175537 time to delete rle : 0.0004894733428955078 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1468 TO DO : save crop sub photo not yet done ! save time : 0.35114479064941406 No data in photo_id : 1382616489 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.20135164260864258 time for calcul the mask position with numpy : 0.07007861137390137 nb_pixel_total : 2056523 time to create 1 rle with new method : 0.1348099708557129 time for calcul the mask position with numpy : 0.007386922836303711 nb_pixel_total : 3946 time to create 1 rle with old method : 0.004417896270751953 time for calcul the mask position with numpy : 0.00657343864440918 nb_pixel_total : 4612 time to create 1 rle with old method : 0.005156755447387695 time for calcul the mask position with numpy : 0.007369041442871094 nb_pixel_total : 8519 time to create 1 rle with old method : 0.009601354598999023 create new chi : 0.25946044921875 time to delete rle : 0.0005509853363037109 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1822 TO DO : save crop sub photo not yet done ! save time : 0.15463852882385254 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.06179332733154297 time for calcul the mask position with numpy : 0.06142997741699219 nb_pixel_total : 1941457 time to create 1 rle with new method : 0.29303431510925293 time for calcul the mask position with numpy : 0.012886524200439453 nb_pixel_total : 24468 time to create 1 rle with old method : 0.028121471405029297 time for calcul the mask position with numpy : 0.009455442428588867 nb_pixel_total : 107675 time to create 1 rle with old method : 0.11835980415344238 create new chi : 0.538470983505249 time to delete rle : 0.0007550716400146484 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2634 TO DO : save crop sub photo not yet done ! save time : 0.28415513038635254 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.10712885856628418 time for calcul the mask position with numpy : 0.27590394020080566 nb_pixel_total : 1939169 time to create 1 rle with new method : 0.10929489135742188 time for calcul the mask position with numpy : 0.007843017578125 nb_pixel_total : 2748 time to create 1 rle with old method : 0.0032303333282470703 time for calcul the mask position with numpy : 0.00903630256652832 nb_pixel_total : 87563 time to create 1 rle with old method : 0.11691451072692871 time for calcul the mask position with numpy : 0.010511159896850586 nb_pixel_total : 44120 time to create 1 rle with old method : 0.05994534492492676 create new chi : 0.6074938774108887 time to delete rle : 0.0007841587066650391 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 2520 TO DO : save crop sub photo not yet done ! save time : 0.32848024368286133 No data in photo_id : 1382616256 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03309154510498047 time for calcul the mask position with numpy : 0.15727472305297852 nb_pixel_total : 1958393 time to create 1 rle with new method : 0.0897526741027832 time for calcul the mask position with numpy : 0.006982326507568359 nb_pixel_total : 115207 time to create 1 rle with old method : 0.12966632843017578 create new chi : 0.3929708003997803 time to delete rle : 0.0003521442413330078 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2152 TO DO : save crop sub photo not yet done ! save time : 0.1758899688720703 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03469228744506836 time for calcul the mask position with numpy : 0.0443422794342041 nb_pixel_total : 1983313 time to create 1 rle with new method : 0.12176084518432617 time for calcul the mask position with numpy : 0.01556706428527832 nb_pixel_total : 90287 time to create 1 rle with old method : 0.10228514671325684 create new chi : 0.29946184158325195 time to delete rle : 0.0004451274871826172 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 2210 TO DO : save crop sub photo not yet done ! save time : 0.33712172508239746 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.15053868293762207 time for calcul the mask position with numpy : 0.17719769477844238 nb_pixel_total : 1951188 time to create 1 rle with new method : 0.29033398628234863 time for calcul the mask position with numpy : 0.007128715515136719 nb_pixel_total : 109490 time to create 1 rle with old method : 0.12350249290466309 time for calcul the mask position with numpy : 0.006876945495605469 nb_pixel_total : 12922 time to create 1 rle with old method : 0.014364480972290039 create new chi : 0.6313519477844238 time to delete rle : 0.00045800209045410156 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 2436 TO DO : save crop sub photo not yet done ! save time : 0.19037175178527832 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.09798765182495117 time for calcul the mask position with numpy : 0.03715872764587402 nb_pixel_total : 1233084 time to create 1 rle with new method : 0.26944518089294434 time for calcul the mask position with numpy : 0.014770984649658203 nb_pixel_total : 814491 time to create 1 rle with new method : 0.2240772247314453 time for calcul the mask position with numpy : 0.005732059478759766 nb_pixel_total : 19674 time to create 1 rle with old method : 0.02075648307800293 time for calcul the mask position with numpy : 0.006124734878540039 nb_pixel_total : 6351 time to create 1 rle with old method : 0.007182121276855469 create new chi : 0.5943753719329834 time to delete rle : 0.0006334781646728516 batch 1 Loaded 7 chid ids of type : 3594 ++++++Number RLEs to save : 4364 TO DO : save crop sub photo not yet done ! save time : 0.46276140213012695 map_output_result : {1382616575: (0.0, 'Should be the crop_list due to order', 0), 1382616560: (0.0, 'Should be the crop_list due to order', 0), 1382616547: (0.0, 'Should be the crop_list due to order', 0), 1382616540: (0.0, 'Should be the crop_list due to order', 0), 1382616530: (0.0, 'Should be the crop_list due to order', 0), 1382616499: (0.0, 'Should be the crop_list due to order', 0.0), 1382616497: (0.0, 'Should be the crop_list due to order', 0.0), 1382616494: (0.0, 'Should be the crop_list due to order', 0), 1382616491: (0.0, 'Should be the crop_list due to order', 0), 1382616489: (0.0, 'Should be the crop_list due to order', 0.0), 1382616486: (0.0, 'Should be the crop_list due to order', 0), 1382616323: (0.0, 'Should be the crop_list due to order', 0), 1382616295: (0.0, 'Should be the crop_list due to order', 0), 1382616256: (0.0, 'Should be the crop_list due to order', 0.0), 1382616228: (0.0, 'Should be the crop_list due to order', 0), 1382616198: (0.0, 'Should be the crop_list due to order', 0), 1382616171: (0.0, 'Should be the crop_list due to order', 0), 1382615800: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1382616575, 1382616560, 1382616547, 1382616540, 1382616530, 1382616499, 1382616497, 1382616494, 1382616491, 1382616489, 1382616486, 1382616323, 1382616295, 1382616256, 1382616228, 1382616198, 1382616171, 1382615800] Looping around the photos to save general results len do output : 18 /1382616575.Didn't retrieve data . /1382616560.Didn't retrieve data . /1382616547.Didn't retrieve data . /1382616540.Didn't retrieve data . /1382616530.Didn't retrieve data . /1382616499.Didn't retrieve data . /1382616497.Didn't retrieve data . /1382616494.Didn't retrieve data . /1382616491.Didn't retrieve data . /1382616489.Didn't retrieve data . /1382616486.Didn't retrieve data . /1382616323.Didn't retrieve data . /1382616295.Didn't retrieve data . /1382616256.Didn't retrieve data . /1382616228.Didn't retrieve data . /1382616198.Didn't retrieve data . /1382616171.Didn't retrieve data . /1382615800.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616575', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616560', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616547', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616540', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616530', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616499', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616497', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616494', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616491', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616489', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616486', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616323', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616295', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616256', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616228', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616198', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616171', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382615800', None, None, None, None, None, '3681349') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 54 time used for this insertion : 0.19977378845214844 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.192340850830078 time spend to save output : 0.20079541206359863 total time spend for step 3 : 14.393136262893677 step4:ventilate_hashtags_in_portfolio Tue Sep 9 00:31:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26666422 get user id for portfolio 26666422 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26666422 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','pet_clair','papier','pehd','metal','autre','background','environnement','mal_croppe','carton','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26666422 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','pet_clair','papier','pehd','metal','autre','background','environnement','mal_croppe','carton','flou')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26666422 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','pet_clair','papier','pehd','metal','autre','background','environnement','mal_croppe','carton','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26666867,26666868,26666869,26666870,26666871,26666872,26666873,26666874,26666875,26666876,26666877?tags=pet_fonce,pet_clair,papier,pehd,metal,autre,background,environnement,mal_croppe,carton,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1382616575, 1382616560, 1382616547, 1382616540, 1382616530, 1382616499, 1382616497, 1382616494, 1382616491, 1382616489, 1382616486, 1382616323, 1382616295, 1382616256, 1382616228, 1382616198, 1382616171, 1382615800] Looping around the photos to save general results len do output : 1 /26666422. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616575', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616560', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616547', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616540', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616530', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616499', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616497', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616494', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616491', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616489', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616486', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616323', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616295', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616256', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616228', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616198', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616171', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382615800', None, None, None, None, None, '3681349') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.019318580627441406 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7224447727203369 time spend to save output : 0.01977849006652832 total time spend for step 4 : 0.7422232627868652 step5:final Tue Sep 9 00:31:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1382616575: ('0.07634344993141291',), 1382616560: ('0.07634344993141291',), 1382616547: ('0.07634344993141291',), 1382616540: ('0.07634344993141291',), 1382616530: ('0.07634344993141291',), 1382616499: ('0.07634344993141291',), 1382616497: ('0.07634344993141291',), 1382616494: ('0.07634344993141291',), 1382616491: ('0.07634344993141291',), 1382616489: ('0.07634344993141291',), 1382616486: ('0.07634344993141291',), 1382616323: ('0.07634344993141291',), 1382616295: ('0.07634344993141291',), 1382616256: ('0.07634344993141291',), 1382616228: ('0.07634344993141291',), 1382616198: ('0.07634344993141291',), 1382616171: ('0.07634344993141291',), 1382615800: ('0.07634344993141291',)} new output for save of step final : {1382616575: ('0.07634344993141291',), 1382616560: ('0.07634344993141291',), 1382616547: ('0.07634344993141291',), 1382616540: ('0.07634344993141291',), 1382616530: ('0.07634344993141291',), 1382616499: ('0.07634344993141291',), 1382616497: ('0.07634344993141291',), 1382616494: ('0.07634344993141291',), 1382616491: ('0.07634344993141291',), 1382616489: ('0.07634344993141291',), 1382616486: ('0.07634344993141291',), 1382616323: ('0.07634344993141291',), 1382616295: ('0.07634344993141291',), 1382616256: ('0.07634344993141291',), 1382616228: ('0.07634344993141291',), 1382616198: ('0.07634344993141291',), 1382616171: ('0.07634344993141291',), 1382615800: ('0.07634344993141291',)} [1382616575, 1382616560, 1382616547, 1382616540, 1382616530, 1382616499, 1382616497, 1382616494, 1382616491, 1382616489, 1382616486, 1382616323, 1382616295, 1382616256, 1382616228, 1382616198, 1382616171, 1382615800] Looping around the photos to save general results len do output : 18 /1382616575.Didn't retrieve data . /1382616560.Didn't retrieve data . /1382616547.Didn't retrieve data . /1382616540.Didn't retrieve data . /1382616530.Didn't retrieve data . /1382616499.Didn't retrieve data . /1382616497.Didn't retrieve data . /1382616494.Didn't retrieve data . /1382616491.Didn't retrieve data . /1382616489.Didn't retrieve data . /1382616486.Didn't retrieve data . /1382616323.Didn't retrieve data . /1382616295.Didn't retrieve data . /1382616256.Didn't retrieve data . /1382616228.Didn't retrieve data . /1382616198.Didn't retrieve data . /1382616171.Didn't retrieve data . /1382615800.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616575', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616560', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616547', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616540', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616530', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616499', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616497', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616494', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616491', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616489', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616486', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616323', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616295', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616256', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616228', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616198', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616171', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382615800', None, None, None, None, None, '3681349') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 54 time used for this insertion : 0.01903557777404785 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12516379356384277 time spend to save output : 0.019838571548461914 total time spend for step 5 : 0.1450023651123047 step6:blur_detection Tue Sep 9 00:31:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1757370630_3827973_1382616575_5f88943a42dfdca215dcdbbcc8a81809.jpg resize: (1080, 1920) 1382616575 -2.311082726812577 treat image : temp/1757370630_3827973_1382616560_deac05885a54b4707a378777d60847bf.jpg resize: (1080, 1920) 1382616560 -1.9757252523008448 treat image : temp/1757370630_3827973_1382616547_8f9b1333b155ecf00264ba117d8edd72.jpg resize: (1080, 1920) 1382616547 -2.0022126241394904 treat image : temp/1757370630_3827973_1382616540_30710991b97b09a881ecd7cf5aa84146.jpg resize: (1080, 1920) 1382616540 -1.9985537591411342 treat image : temp/1757370630_3827973_1382616530_69f0d7ad181332fec7f2e0bc74083e50.jpg resize: (1080, 1920) 1382616530 -2.5747911195012176 treat image : temp/1757370630_3827973_1382616499_cfc0d55633f511be6f944ab4ac8fb78e.jpg resize: (1080, 1920) 1382616499 -2.79696634799474 treat image : temp/1757370630_3827973_1382616497_2ac8ea47ab8f0347f1f723506fa3add6.jpg resize: (1080, 1920) 1382616497 -2.4095534640937637 treat image : temp/1757370630_3827973_1382616494_9b70ae36d7ede57bcf18a857228900a9.jpg resize: (1080, 1920) 1382616494 -1.9195332241753096 treat image : temp/1757370630_3827973_1382616491_c087fbc26ff01a773ae1e1e89125f712.jpg resize: (1080, 1920) 1382616491 -2.666579839702004 treat image : temp/1757370630_3827973_1382616489_f0f06d6b99da486a18ac6661daba2c87.jpg resize: (1080, 1920) 1382616489 -2.479212797828019 treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99.jpg resize: (1080, 1920) 1382616486 -0.8290098138955787 treat image : temp/1757370630_3827973_1382616323_39ce0b076da0575addab6ca052d10b54.jpg resize: (1080, 1920) 1382616323 -1.9543896836471033 treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7.jpg resize: (1080, 1920) 1382616295 -1.577123048821406 treat image : temp/1757370630_3827973_1382616256_9c18bf751fce798fe4bbb497ee160af2.jpg resize: (1080, 1920) 1382616256 -1.8677094450448204 treat image : temp/1757370630_3827973_1382616228_f467ecd6b0a5f7c971873d0fd60535bb.jpg resize: (1080, 1920) 1382616228 -2.8624193051518496 treat image : temp/1757370630_3827973_1382616198_5267d171582abab2283f170d7768116c.jpg resize: (1080, 1920) 1382616198 -2.224699706530211 treat image : temp/1757370630_3827973_1382616171_b24305284145f27a72ca13709cf933a5.jpg resize: (1080, 1920) 1382616171 -2.6731050610675493 treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab.jpg resize: (1080, 1920) 1382615800 -1.3912473495836346 treat image : temp/1757370630_3827973_1382616575_5f88943a42dfdca215dcdbbcc8a81809_rle_crop_3951907458_0.png resize: (111, 101) 1382701183 -3.4267141263380787 treat image : temp/1757370630_3827973_1382616547_8f9b1333b155ecf00264ba117d8edd72_rle_crop_3951907461_0.png resize: (145, 256) 1382701184 -0.17899939432849354 treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99_rle_crop_3951907469_0.png resize: (114, 46) 1382701185 1.6839037633040081 treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99_rle_crop_3951907468_0.png resize: (110, 54) 1382701186 0.7764352983005187 treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab_rle_crop_3951907479_0.png resize: (98, 95) 1382701187 4.200755903012236 treat image : temp/1757370630_3827973_1382616575_5f88943a42dfdca215dcdbbcc8a81809_rle_crop_3951907457_0.png resize: (968, 871) 1382701188 -0.32330714236280533 treat image : temp/1757370630_3827973_1382616560_deac05885a54b4707a378777d60847bf_rle_crop_3951907459_0.png resize: (73, 50) 1382701189 3.3981957410591352 treat image : temp/1757370630_3827973_1382616171_b24305284145f27a72ca13709cf933a5_rle_crop_3951907477_0.png resize: (144, 120) 1382701190 -0.4977790069881275 treat image : temp/1757370630_3827973_1382616560_deac05885a54b4707a378777d60847bf_rle_crop_3951907460_0.png resize: (967, 1100) 1382701196 -2.4430654872967126 treat image : temp/1757370630_3827973_1382616540_30710991b97b09a881ecd7cf5aa84146_rle_crop_3951907462_0.png resize: (480, 347) 1382701197 0.2047792609354097 treat image : temp/1757370630_3827973_1382616530_69f0d7ad181332fec7f2e0bc74083e50_rle_crop_3951907463_0.png resize: (124, 107) 1382701198 -2.4910577323103533 treat image : temp/1757370630_3827973_1382616530_69f0d7ad181332fec7f2e0bc74083e50_rle_crop_3951907464_0.png resize: (42, 110) 1382701199 0.11774859971647492 treat image : temp/1757370630_3827973_1382616494_9b70ae36d7ede57bcf18a857228900a9_rle_crop_3951907465_0.png resize: (176, 150) 1382701200 -2.1736438416171704 treat image : temp/1757370630_3827973_1382616491_c087fbc26ff01a773ae1e1e89125f712_rle_crop_3951907466_0.png resize: (173, 101) 1382701201 -3.617480238341936 treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99_rle_crop_3951907467_0.png resize: (142, 70) 1382701202 -1.359149458843723 treat image : temp/1757370630_3827973_1382616323_39ce0b076da0575addab6ca052d10b54_rle_crop_3951907470_0.png resize: (515, 345) 1382701203 0.16818552909545284 treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7_rle_crop_3951907472_0.png resize: (158, 353) 1382701204 -2.826187349993192 treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7_rle_crop_3951907474_0.png resize: (74, 51) 1382701205 -2.0731144375780914 treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7_rle_crop_3951907473_0.png resize: (470, 337) 1382701206 -0.44777055663286464 treat image : temp/1757370630_3827973_1382616228_f467ecd6b0a5f7c971873d0fd60535bb_rle_crop_3951907475_0.png resize: (527, 391) 1382701207 -0.07932589803479644 treat image : temp/1757370630_3827973_1382616198_5267d171582abab2283f170d7768116c_rle_crop_3951907476_0.png resize: (497, 336) 1382701208 -0.6738929310921665 treat image : temp/1757370630_3827973_1382616171_b24305284145f27a72ca13709cf933a5_rle_crop_3951907478_0.png resize: (519, 339) 1382701209 -0.21072080796486062 treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab_rle_crop_3951907481_0.png resize: (958, 1147) 1382701210 -1.1752929861756132 treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab_rle_crop_3951907480_0.png resize: (177, 155) 1382701211 -0.9092951019274647 treat image : temp/1757370630_3827973_1382616323_39ce0b076da0575addab6ca052d10b54_rle_crop_3951907471_0.png resize: (256, 126) 1382701214 -0.9901653171983901 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 43 time used for this insertion : 0.014196157455444336 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 43 time used for this insertion : 0.6097564697265625 save missing photos in datou_result : time spend for datou_step_exec : 15.475239515304565 time spend to save output : 0.628469705581665 total time spend for step 6 : 16.10370922088623 step7:brightness Tue Sep 9 00:31:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1757370630_3827973_1382616575_5f88943a42dfdca215dcdbbcc8a81809.jpg treat image : temp/1757370630_3827973_1382616560_deac05885a54b4707a378777d60847bf.jpg treat image : temp/1757370630_3827973_1382616547_8f9b1333b155ecf00264ba117d8edd72.jpg treat image : temp/1757370630_3827973_1382616540_30710991b97b09a881ecd7cf5aa84146.jpg treat image : temp/1757370630_3827973_1382616530_69f0d7ad181332fec7f2e0bc74083e50.jpg treat image : temp/1757370630_3827973_1382616499_cfc0d55633f511be6f944ab4ac8fb78e.jpg treat image : temp/1757370630_3827973_1382616497_2ac8ea47ab8f0347f1f723506fa3add6.jpg treat image : temp/1757370630_3827973_1382616494_9b70ae36d7ede57bcf18a857228900a9.jpg treat image : temp/1757370630_3827973_1382616491_c087fbc26ff01a773ae1e1e89125f712.jpg treat image : temp/1757370630_3827973_1382616489_f0f06d6b99da486a18ac6661daba2c87.jpg treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99.jpg treat image : temp/1757370630_3827973_1382616323_39ce0b076da0575addab6ca052d10b54.jpg treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7.jpg treat image : temp/1757370630_3827973_1382616256_9c18bf751fce798fe4bbb497ee160af2.jpg treat image : temp/1757370630_3827973_1382616228_f467ecd6b0a5f7c971873d0fd60535bb.jpg treat image : temp/1757370630_3827973_1382616198_5267d171582abab2283f170d7768116c.jpg treat image : temp/1757370630_3827973_1382616171_b24305284145f27a72ca13709cf933a5.jpg treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab.jpg treat image : temp/1757370630_3827973_1382616575_5f88943a42dfdca215dcdbbcc8a81809_rle_crop_3951907458_0.png treat image : temp/1757370630_3827973_1382616547_8f9b1333b155ecf00264ba117d8edd72_rle_crop_3951907461_0.png treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99_rle_crop_3951907469_0.png treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99_rle_crop_3951907468_0.png treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab_rle_crop_3951907479_0.png treat image : temp/1757370630_3827973_1382616575_5f88943a42dfdca215dcdbbcc8a81809_rle_crop_3951907457_0.png treat image : temp/1757370630_3827973_1382616560_deac05885a54b4707a378777d60847bf_rle_crop_3951907459_0.png treat image : temp/1757370630_3827973_1382616171_b24305284145f27a72ca13709cf933a5_rle_crop_3951907477_0.png treat image : temp/1757370630_3827973_1382616560_deac05885a54b4707a378777d60847bf_rle_crop_3951907460_0.png treat image : temp/1757370630_3827973_1382616540_30710991b97b09a881ecd7cf5aa84146_rle_crop_3951907462_0.png treat image : temp/1757370630_3827973_1382616530_69f0d7ad181332fec7f2e0bc74083e50_rle_crop_3951907463_0.png treat image : temp/1757370630_3827973_1382616530_69f0d7ad181332fec7f2e0bc74083e50_rle_crop_3951907464_0.png treat image : temp/1757370630_3827973_1382616494_9b70ae36d7ede57bcf18a857228900a9_rle_crop_3951907465_0.png treat image : temp/1757370630_3827973_1382616491_c087fbc26ff01a773ae1e1e89125f712_rle_crop_3951907466_0.png treat image : temp/1757370630_3827973_1382616486_92cb10a18670a34baaf1cd0fc45c8c99_rle_crop_3951907467_0.png treat image : temp/1757370630_3827973_1382616323_39ce0b076da0575addab6ca052d10b54_rle_crop_3951907470_0.png treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7_rle_crop_3951907472_0.png treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7_rle_crop_3951907474_0.png treat image : temp/1757370630_3827973_1382616295_e801f447097f08e74906e9e87ad27fb7_rle_crop_3951907473_0.png treat image : temp/1757370630_3827973_1382616228_f467ecd6b0a5f7c971873d0fd60535bb_rle_crop_3951907475_0.png treat image : temp/1757370630_3827973_1382616198_5267d171582abab2283f170d7768116c_rle_crop_3951907476_0.png treat image : temp/1757370630_3827973_1382616171_b24305284145f27a72ca13709cf933a5_rle_crop_3951907478_0.png treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab_rle_crop_3951907481_0.png treat image : temp/1757370630_3827973_1382615800_ec815a9f54c60950316c27c25dac44ab_rle_crop_3951907480_0.png treat image : temp/1757370630_3827973_1382616323_39ce0b076da0575addab6ca052d10b54_rle_crop_3951907471_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 43 time used for this insertion : 0.016218900680541992 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 43 time used for this insertion : 0.2834892272949219 save missing photos in datou_result : time spend for datou_step_exec : 4.279789209365845 time spend to save output : 0.30534863471984863 total time spend for step 7 : 4.585137844085693 step8:velours_tree Tue Sep 9 00:32:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.18279337882995605 time spend to save output : 4.553794860839844e-05 total time spend for step 8 : 0.18283891677856445 step9:send_mail_cod Tue Sep 9 00:32:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26666422_09-09-2025_00_32_00.pdf 26666867 imagette266668671757370720 26666868 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette266668681757370720 26666869 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette266668691757370721 26666870 imagette266668701757370722 26666871 imagette266668711757370722 26666872 change filename to text .imagette266668721757370722 26666873 imagette266668731757370722 26666875 imagette266668751757370722 26666876 change filename to text .change filename to text .change filename to text .imagette266668761757370722 26666877 imagette266668771757370722 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26666422 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26666867,26666868,26666869,26666870,26666871,26666872,26666873,26666874,26666875,26666876,26666877?tags=pet_fonce,pet_clair,papier,pehd,metal,autre,background,environnement,mal_croppe,carton,flou args[1382616575] : ((1382616575, -2.311082726812577, 492609224), (1382616575, 0.6208920597493293, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616560] : ((1382616560, -1.9757252523008448, 492688767), (1382616560, 0.40212268839510074, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616547] : ((1382616547, -2.0022126241394904, 492609224), (1382616547, 0.6270581645642758, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616540] : ((1382616540, -1.9985537591411342, 492688767), (1382616540, 0.5792396650467218, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616530] : ((1382616530, -2.5747911195012176, 492609224), (1382616530, 0.46243274168838494, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616499] : ((1382616499, -2.79696634799474, 492609224), (1382616499, 0.6944968128167076, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616497] : ((1382616497, -2.4095534640937637, 492609224), (1382616497, 0.564734191750847, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616494] : ((1382616494, -1.9195332241753096, 492688767), (1382616494, 0.662873889320119, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616491] : ((1382616491, -2.666579839702004, 492609224), (1382616491, 0.562625753791752, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616489] : ((1382616489, -2.479212797828019, 492609224), (1382616489, 0.5566667262257694, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616486] : ((1382616486, -0.8290098138955787, 492688767), (1382616486, 0.5359530372855037, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616323] : ((1382616323, -1.9543896836471033, 492688767), (1382616323, 0.8238809670545011, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616295] : ((1382616295, -1.577123048821406, 492688767), (1382616295, 0.6501645254732263, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616256] : ((1382616256, -1.8677094450448204, 492688767), (1382616256, 0.6263671541249686, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616228] : ((1382616228, -2.8624193051518496, 492609224), (1382616228, 0.6150847454979075, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616198] : ((1382616198, -2.224699706530211, 492609224), (1382616198, 0.6722139535475168, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382616171] : ((1382616171, -2.6731050610675493, 492609224), (1382616171, 0.6638772715282476, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com args[1382615800] : ((1382615800, -1.3912473495836346, 492688767), (1382615800, 0.7050564872342239, 2107752395), '0.07634344993141291') We are sending mail with results at report@fotonower.com refus_total : 0.07634344993141291 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26666422 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26666422_09-09-2025_00_32_00.pdf results_Auto_P26666422_09-09-2025_00_32_00.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26666422_09-09-2025_00_32_00.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26666422','results_Auto_P26666422_09-09-2025_00_32_00.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26666422_09-09-2025_00_32_00.pdf','pdf','','0.43','0.07634344993141291') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26666422

https://www.fotonower.com/image?json=false&list_photos_id=1382616575
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616560
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616547
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616540
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616530
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616499
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616497
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616494
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616491
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616489
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616486
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616323
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616295
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616256
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616228
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616198
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382616171
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1382615800
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 7.63%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/26666868?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26666869?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/26666872?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26666876?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26666422_09-09-2025_00_32_00.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/26666867,26666868,26666869,26666870,26666871,26666872,26666873,26666874,26666875,26666876,26666877?tags=pet_fonce,pet_clair,papier,pehd,metal,autre,background,environnement,mal_croppe,carton,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 08 Sep 2025 22:32:05 GMT Content-Length: 0 Connection: close X-Message-Id: lxp68RqRTWiMHsBLiyg3VA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1382616575, 1382616560, 1382616547, 1382616540, 1382616530, 1382616499, 1382616497, 1382616494, 1382616491, 1382616489, 1382616486, 1382616323, 1382616295, 1382616256, 1382616228, 1382616198, 1382616171, 1382615800] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616575', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616560', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616547', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616540', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616530', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616499', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616497', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616494', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616491', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616489', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616486', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616323', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616295', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616256', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616228', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616198', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616171', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382615800', None, None, None, None, None, '3681349') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.020128250122070312 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.858842611312866 time spend to save output : 0.020405292510986328 total time spend for step 9 : 4.8792479038238525 step10:split_time_score Tue Sep 9 00:32:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('19', 18),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 08092025 26666422 Nombre de photos uploadées : 18 / 23040 (0%) 08092025 26666422 Nombre de photos taguées (types de déchets): 0 / 18 (0%) 08092025 26666422 Nombre de photos taguées (volume) : 0 / 18 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 7.62939453125e-06 ?????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0009627342224121094 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.20002031326293945 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.1588576630015432 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26629339_08-09-2025_08_56_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26629339 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26629339 AND mptpi.`type`=3594 To do Qualite : 0.21931737075617283 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26629340_08-09-2025_08_52_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26629340 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26629340 AND mptpi.`type`=3594 To do Qualite : 0.17503380155723905 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26629342_08-09-2025_08_41_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26629342 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26629342 AND mptpi.`type`=3594 To do Qualite : 0.04856762322984748 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26631054_08-09-2025_09_42_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26631054 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26631054 AND mptpi.`type`=3594 To do Qualite : 0.12406641830632716 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26638960_08-09-2025_12_51_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26638960 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26638960 AND mptpi.`type`=3594 To do Qualite : 0.0 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26638961_08-09-2025_13_41_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26638961 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26638961 AND mptpi.`type`=3594 To do Qualite : 0.08286177749719416 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26639467_08-09-2025_13_31_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26639467 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26639467 AND mptpi.`type`=3594 To do Qualite : 0.05007662150508353 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26639469_08-09-2025_13_21_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26639469 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26639469 AND mptpi.`type`=3594 To do Qualite : 0.05256312692901231 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26649069_08-09-2025_17_52_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26649069 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26649069 AND mptpi.`type`=3594 To do Qualite : 0.082544849537037 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26658004_08-09-2025_21_30_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26658004 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26658004 AND mptpi.`type`=3594 To do Qualite : 0.07342282572751324 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26649947_08-09-2025_18_11_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26649947 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26649947 AND mptpi.`type`=3594 To do Qualite : 0.11211073133680559 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26649074_08-09-2025_17_32_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26649074 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26649074 AND mptpi.`type`=3594 To do Qualite : 0.03960919415509259 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26649949_08-09-2025_19_11_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26649949 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26649949 AND mptpi.`type`=3594 To do Qualite : 0.08574476650563605 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26658027_08-09-2025_22_22_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26658027 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26658027 AND mptpi.`type`=3594 To do Qualite : 0.07634344993141291 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26666422_09-09-2025_00_32_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26666422 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26666422 AND mptpi.`type`=3594 To do Qualite : 0.06465544871794869 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26666423_09-09-2025_00_21_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26666423 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26666423 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'08092025': {'nb_upload': 18, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1382616575, 1382616560, 1382616547, 1382616540, 1382616530, 1382616499, 1382616497, 1382616494, 1382616491, 1382616489, 1382616486, 1382616323, 1382616295, 1382616256, 1382616228, 1382616198, 1382616171, 1382615800] Looping around the photos to save general results len do output : 1 /26666422Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616575', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616560', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616547', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616540', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616530', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616499', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616497', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616494', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616491', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616489', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616486', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616323', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616295', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616256', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616228', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616198', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382616171', None, None, None, None, None, '3681349') ('3318', None, None, None, None, None, None, None, '3681349') ('3318', '26666422', '1382615800', None, None, None, None, None, '3681349') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.020552396774291992 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6504390239715576 time spend to save output : 0.02086806297302246 total time spend for step 10 : 1.67130708694458 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 18 set_done_treatment 56.65user 26.86system 1:40.94elapsed 82%CPU (0avgtext+0avgdata 2875388maxresident)k 524872inputs+29960outputs (2323major+2116329minor)pagefaults 0swaps