python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 479308 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3742863'] with mtr_portfolio_ids : ['27012373'] and first list_photo_ids : [] new path : /proc/479308/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 13 ; length of list_pids : 13 ; length of list_args : 13 time to download the photos : 1.806401014328003 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu Sep 18 22:00:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10434 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-18 22:00:36.071618: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-18 22:00:36.096551: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-18 22:00:36.098125: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2b68000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-18 22:00:36.098173: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-18 22:00:36.100763: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-18 22:00:36.201942: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1b028160 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-18 22:00:36.202016: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-18 22:00:36.203638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-18 22:00:36.204055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-18 22:00:36.207304: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-18 22:00:36.210484: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-18 22:00:36.211172: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-18 22:00:36.214038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-18 22:00:36.215155: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-18 22:00:36.219601: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-18 22:00:36.221145: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-18 22:00:36.221238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-18 22:00:36.221989: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-18 22:00:36.222005: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-18 22:00:36.222014: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-18 22:00:36.223299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9665 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-18 22:00:36.525537: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-18 22:00:36.525639: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-18 22:00:36.525668: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-18 22:00:36.525693: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-18 22:00:36.525717: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-18 22:00:36.525741: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-18 22:00:36.525766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-18 22:00:36.525790: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-18 22:00:36.527939: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-18 22:00:36.529766: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-18 22:00:36.529842: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-18 22:00:36.529859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-18 22:00:36.529873: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-18 22:00:36.529887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-18 22:00:36.529905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-18 22:00:36.529919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-18 22:00:36.529934: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-18 22:00:36.531222: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-18 22:00:36.531257: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-18 22:00:36.531266: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-18 22:00:36.531274: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-18 22:00:36.532639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9665 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-18 22:00:45.279808: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-18 22:00:45.478314: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 13 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 23.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 479943 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 684 tf kernel not reseted sub process len(results) : 13 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 13 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1521 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0003199577331542969 nb_pixel_total : 8538 time to create 1 rle with old method : 0.011025190353393555 length of segment : 171 time for calcul the mask position with numpy : 0.0004012584686279297 nb_pixel_total : 17888 time to create 1 rle with old method : 0.020644187927246094 length of segment : 180 time for calcul the mask position with numpy : 0.0025005340576171875 nb_pixel_total : 137444 time to create 1 rle with old method : 0.16520214080810547 length of segment : 612 time for calcul the mask position with numpy : 0.00022745132446289062 nb_pixel_total : 10330 time to create 1 rle with old method : 0.012060403823852539 length of segment : 116 time for calcul the mask position with numpy : 0.0007734298706054688 nb_pixel_total : 33406 time to create 1 rle with old method : 0.04583001136779785 length of segment : 218 time for calcul the mask position with numpy : 0.00024175643920898438 nb_pixel_total : 8706 time to create 1 rle with old method : 0.010286092758178711 length of segment : 160 time for calcul the mask position with numpy : 0.0002505779266357422 nb_pixel_total : 9453 time to create 1 rle with old method : 0.011325836181640625 length of segment : 154 time for calcul the mask position with numpy : 0.00026226043701171875 nb_pixel_total : 10830 time to create 1 rle with old method : 0.012878656387329102 length of segment : 202 time for calcul the mask position with numpy : 0.0002522468566894531 nb_pixel_total : 10181 time to create 1 rle with old method : 0.011924028396606445 length of segment : 164 time for calcul the mask position with numpy : 0.00020265579223632812 nb_pixel_total : 9289 time to create 1 rle with old method : 0.011003971099853516 length of segment : 110 time for calcul the mask position with numpy : 0.0001583099365234375 nb_pixel_total : 6235 time to create 1 rle with old method : 0.00749969482421875 length of segment : 105 time for calcul the mask position with numpy : 0.00011014938354492188 nb_pixel_total : 5171 time to create 1 rle with old method : 0.006488323211669922 length of segment : 90 time for calcul the mask position with numpy : 0.09842276573181152 nb_pixel_total : 1491621 time to create 1 rle with new method : 0.23907136917114258 length of segment : 1130 time for calcul the mask position with numpy : 0.0001876354217529297 nb_pixel_total : 8740 time to create 1 rle with old method : 0.01080179214477539 length of segment : 118 time for calcul the mask position with numpy : 0.00011515617370605469 nb_pixel_total : 1938 time to create 1 rle with old method : 0.0032317638397216797 length of segment : 41 time for calcul the mask position with numpy : 0.00015878677368164062 nb_pixel_total : 8599 time to create 1 rle with old method : 0.009945392608642578 length of segment : 116 time for calcul the mask position with numpy : 0.0002875328063964844 nb_pixel_total : 8776 time to create 1 rle with old method : 0.010448217391967773 length of segment : 162 time for calcul the mask position with numpy : 0.019925594329833984 nb_pixel_total : 1578707 time to create 1 rle with new method : 0.07422637939453125 length of segment : 1212 time for calcul the mask position with numpy : 0.00026988983154296875 nb_pixel_total : 11029 time to create 1 rle with old method : 0.01481938362121582 length of segment : 84 time for calcul the mask position with numpy : 0.00013256072998046875 nb_pixel_total : 7785 time to create 1 rle with old method : 0.009299278259277344 length of segment : 62 time for calcul the mask position with numpy : 0.00020766258239746094 nb_pixel_total : 5470 time to create 1 rle with old method : 0.006450176239013672 length of segment : 111 time for calcul the mask position with numpy : 0.0025060176849365234 nb_pixel_total : 103017 time to create 1 rle with old method : 0.11127209663391113 length of segment : 544 time for calcul the mask position with numpy : 0.0002665519714355469 nb_pixel_total : 8016 time to create 1 rle with old method : 0.009386777877807617 length of segment : 166 time for calcul the mask position with numpy : 0.0267031192779541 nb_pixel_total : 1259983 time to create 1 rle with new method : 0.0735926628112793 length of segment : 1430 time for calcul the mask position with numpy : 0.00032901763916015625 nb_pixel_total : 8789 time to create 1 rle with old method : 0.010830879211425781 length of segment : 126 time for calcul the mask position with numpy : 0.0003902912139892578 nb_pixel_total : 9203 time to create 1 rle with old method : 0.010462522506713867 length of segment : 161 time for calcul the mask position with numpy : 0.001911163330078125 nb_pixel_total : 114674 time to create 1 rle with old method : 0.1278836727142334 length of segment : 512 time for calcul the mask position with numpy : 0.009986400604248047 nb_pixel_total : 680366 time to create 1 rle with new method : 0.03360414505004883 length of segment : 1059 time for calcul the mask position with numpy : 0.0016717910766601562 nb_pixel_total : 90602 time to create 1 rle with old method : 0.10643696784973145 length of segment : 502 time for calcul the mask position with numpy : 0.0003008842468261719 nb_pixel_total : 8493 time to create 1 rle with old method : 0.01018977165222168 length of segment : 177 time for calcul the mask position with numpy : 0.00041103363037109375 nb_pixel_total : 9771 time to create 1 rle with old method : 0.011519193649291992 length of segment : 185 time spent for convertir_results : 3.645841121673584 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 31 chid ids of type : 3594 Number RLEs to save : 10180 save missing photos in datou_result : time spend for datou_step_exec : 27.483064889907837 time spend to save output : 0.5959804058074951 total time spend for step 1 : 28.079045295715332 step2:crop_condition Thu Sep 18 22:01:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 13 ! batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758225662_479308 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.421599864959717 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3736932 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758225667_479308 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.380988597869873 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3736932 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758225691_479308 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 12 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.358356475830078 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1384932086, 1384931925, 1384931924, 1384931908, 1384931878, 1384931853, 1384931826, 1384931542, 1384931517, 1384931488, 1384931464, 1384931439, 1384931414] Looping around the photos to save general results len do output : 31 /1384946170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946178Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946183Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384946237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384932086', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931925', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931924', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931908', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931878', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931853', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931826', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931542', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931517', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931488', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931464', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931439', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931414', None, None, None, None, None, '3742863') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 106 time used for this insertion : 0.018617868423461914 save_final save missing photos in datou_result : time spend for datou_step_exec : 33.025943994522095 time spend to save output : 0.02007126808166504 total time spend for step 2 : 33.04601526260376 step3:rle_unique_nms_with_priority Thu Sep 18 22:01:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 31 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++ No data in photo_id : 1384932086 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.3084530830383301 time for calcul the mask position with numpy : 0.1101229190826416 nb_pixel_total : 1909730 time to create 1 rle with new method : 0.07618451118469238 time for calcul the mask position with numpy : 0.006482839584350586 nb_pixel_total : 137444 time to create 1 rle with old method : 0.14517498016357422 time for calcul the mask position with numpy : 0.006154537200927734 nb_pixel_total : 17888 time to create 1 rle with old method : 0.022217512130737305 time for calcul the mask position with numpy : 0.005837440490722656 nb_pixel_total : 8538 time to create 1 rle with old method : 0.009259939193725586 create new chi : 0.3918614387512207 time to delete rle : 0.033203840255737305 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 3006 TO DO : save crop sub photo not yet done ! save time : 0.21314644813537598 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.22225260734558105 time for calcul the mask position with numpy : 0.09182095527648926 nb_pixel_total : 2029864 time to create 1 rle with new method : 0.23976492881774902 time for calcul the mask position with numpy : 0.0059871673583984375 nb_pixel_total : 33406 time to create 1 rle with old method : 0.036260128021240234 time for calcul the mask position with numpy : 0.006253957748413086 nb_pixel_total : 10330 time to create 1 rle with old method : 0.010796308517456055 create new chi : 0.3942444324493408 time to delete rle : 0.0002791881561279297 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1748 TO DO : save crop sub photo not yet done ! save time : 0.14113354682922363 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.07260704040527344 time for calcul the mask position with numpy : 0.059969186782836914 nb_pixel_total : 2064894 time to create 1 rle with new method : 0.18011093139648438 time for calcul the mask position with numpy : 0.005832195281982422 nb_pixel_total : 8706 time to create 1 rle with old method : 0.009213447570800781 create new chi : 0.2585935592651367 time to delete rle : 0.00027751922607421875 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1400 TO DO : save crop sub photo not yet done ! save time : 0.11088728904724121 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03801465034484863 time for calcul the mask position with numpy : 0.14603376388549805 nb_pixel_total : 2064147 time to create 1 rle with new method : 0.09004473686218262 time for calcul the mask position with numpy : 0.006554365158081055 nb_pixel_total : 9453 time to create 1 rle with old method : 0.009774923324584961 create new chi : 0.2641615867614746 time to delete rle : 0.0003209114074707031 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1388 TO DO : save crop sub photo not yet done ! save time : 0.1158149242401123 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.19681620597839355 time for calcul the mask position with numpy : 0.1262834072113037 nb_pixel_total : 2037065 time to create 1 rle with new method : 0.19703936576843262 time for calcul the mask position with numpy : 0.006607770919799805 nb_pixel_total : 6235 time to create 1 rle with old method : 0.006388664245605469 time for calcul the mask position with numpy : 0.0064678192138671875 nb_pixel_total : 9289 time to create 1 rle with old method : 0.009540319442749023 time for calcul the mask position with numpy : 0.006345272064208984 nb_pixel_total : 10181 time to create 1 rle with old method : 0.010372400283813477 time for calcul the mask position with numpy : 0.006752729415893555 nb_pixel_total : 10830 time to create 1 rle with old method : 0.011064291000366211 create new chi : 0.3961188793182373 time to delete rle : 0.0004799365997314453 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 2242 TO DO : save crop sub photo not yet done ! save time : 0.17525124549865723 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.05376243591308594 time for calcul the mask position with numpy : 0.00997471809387207 nb_pixel_total : 581979 time to create 1 rle with new method : 0.07627606391906738 time for calcul the mask position with numpy : 0.020464658737182617 nb_pixel_total : 1486450 time to create 1 rle with new method : 0.16264009475708008 time for calcul the mask position with numpy : 0.005787372589111328 nb_pixel_total : 5171 time to create 1 rle with old method : 0.0056231021881103516 create new chi : 0.29405665397644043 time to delete rle : 0.0005145072937011719 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 3520 TO DO : save crop sub photo not yet done ! save time : 0.2272329330444336 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.08455824851989746 time for calcul the mask position with numpy : 0.010217666625976562 nb_pixel_total : 478332 time to create 1 rle with new method : 0.15844225883483887 time for calcul the mask position with numpy : 0.0057926177978515625 nb_pixel_total : 7785 time to create 1 rle with old method : 0.008223295211791992 time for calcul the mask position with numpy : 0.11927103996276855 nb_pixel_total : 1570108 time to create 1 rle with new method : 0.07185173034667969 time for calcul the mask position with numpy : 0.005769252777099609 nb_pixel_total : 8776 time to create 1 rle with old method : 0.009058237075805664 time for calcul the mask position with numpy : 0.006052732467651367 nb_pixel_total : 8599 time to create 1 rle with old method : 0.008962392807006836 create new chi : 0.41789865493774414 time to delete rle : 0.0005290508270263672 batch 1 Loaded 10 chid ids of type : 3594 +++++Number RLEs to save : 4184 TO DO : save crop sub photo not yet done ! save time : 0.2757875919342041 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.061577796936035156 time for calcul the mask position with numpy : 0.022595882415771484 nb_pixel_total : 702584 time to create 1 rle with new method : 0.1671605110168457 time for calcul the mask position with numpy : 0.01636219024658203 nb_pixel_total : 1254513 time to create 1 rle with new method : 0.1555004119873047 time for calcul the mask position with numpy : 0.0060346126556396484 nb_pixel_total : 8016 time to create 1 rle with old method : 0.009046077728271484 time for calcul the mask position with numpy : 0.0064694881439208984 nb_pixel_total : 103017 time to create 1 rle with old method : 0.11293721199035645 time for calcul the mask position with numpy : 0.0059375762939453125 nb_pixel_total : 5470 time to create 1 rle with old method : 0.006109714508056641 create new chi : 0.5245788097381592 time to delete rle : 0.0006842613220214844 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 5580 TO DO : save crop sub photo not yet done ! save time : 0.3506009578704834 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03276872634887695 time for calcul the mask position with numpy : 0.019369840621948242 nb_pixel_total : 2064811 time to create 1 rle with new method : 0.07362675666809082 time for calcul the mask position with numpy : 0.006437063217163086 nb_pixel_total : 8789 time to create 1 rle with old method : 0.009639978408813477 create new chi : 0.10930204391479492 time to delete rle : 0.00021266937255859375 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1332 TO DO : save crop sub photo not yet done ! save time : 0.1128239631652832 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.16663169860839844 time for calcul the mask position with numpy : 0.10458993911743164 nb_pixel_total : 1949723 time to create 1 rle with new method : 0.15596556663513184 time for calcul the mask position with numpy : 0.006535530090332031 nb_pixel_total : 114674 time to create 1 rle with old method : 0.12154412269592285 time for calcul the mask position with numpy : 0.0060999393463134766 nb_pixel_total : 9203 time to create 1 rle with old method : 0.01011800765991211 create new chi : 0.41462087631225586 time to delete rle : 0.0004870891571044922 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2426 TO DO : save crop sub photo not yet done ! save time : 0.18140196800231934 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.06899595260620117 time for calcul the mask position with numpy : 0.04509234428405762 nb_pixel_total : 1294139 time to create 1 rle with new method : 0.14542913436889648 time for calcul the mask position with numpy : 0.009722471237182617 nb_pixel_total : 8493 time to create 1 rle with old method : 0.00934457778930664 time for calcul the mask position with numpy : 0.009885787963867188 nb_pixel_total : 90602 time to create 1 rle with old method : 0.09629607200622559 time for calcul the mask position with numpy : 0.01056981086730957 nb_pixel_total : 680366 time to create 1 rle with new method : 0.15950274467468262 create new chi : 0.5014088153839111 time to delete rle : 0.0006413459777832031 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 4556 TO DO : save crop sub photo not yet done ! save time : 0.3008105754852295 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.06540966033935547 time for calcul the mask position with numpy : 0.026925086975097656 nb_pixel_total : 2063829 time to create 1 rle with new method : 0.08485102653503418 time for calcul the mask position with numpy : 0.006055355072021484 nb_pixel_total : 9771 time to create 1 rle with old method : 0.010805130004882812 create new chi : 0.13548636436462402 time to delete rle : 0.00036406517028808594 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1450 TO DO : save crop sub photo not yet done ! save time : 0.11886858940124512 map_output_result : {1384932086: (0.0, 'Should be the crop_list due to order', 0.0), 1384931925: (0.0, 'Should be the crop_list due to order', 0), 1384931924: (0.0, 'Should be the crop_list due to order', 0), 1384931908: (0.0, 'Should be the crop_list due to order', 0), 1384931878: (0.0, 'Should be the crop_list due to order', 0), 1384931853: (0.0, 'Should be the crop_list due to order', 0), 1384931826: (0.0, 'Should be the crop_list due to order', 0), 1384931542: (0.0, 'Should be the crop_list due to order', 0), 1384931517: (0.0, 'Should be the crop_list due to order', 0), 1384931488: (0.0, 'Should be the crop_list due to order', 0), 1384931464: (0.0, 'Should be the crop_list due to order', 0), 1384931439: (0.0, 'Should be the crop_list due to order', 0), 1384931414: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1384932086, 1384931925, 1384931924, 1384931908, 1384931878, 1384931853, 1384931826, 1384931542, 1384931517, 1384931488, 1384931464, 1384931439, 1384931414] Looping around the photos to save general results len do output : 13 /1384932086.Didn't retrieve data . /1384931925.Didn't retrieve data . /1384931924.Didn't retrieve data . /1384931908.Didn't retrieve data . /1384931878.Didn't retrieve data . /1384931853.Didn't retrieve data . /1384931826.Didn't retrieve data . /1384931542.Didn't retrieve data . /1384931517.Didn't retrieve data . /1384931488.Didn't retrieve data . /1384931464.Didn't retrieve data . /1384931439.Didn't retrieve data . /1384931414.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384932086', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931925', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931924', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931908', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931878', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931853', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931826', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931542', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931517', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931488', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931464', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931439', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931414', None, None, None, None, None, '3742863') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 39 time used for this insertion : 0.014701366424560547 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.255438089370728 time spend to save output : 0.015193700790405273 total time spend for step 3 : 8.270631790161133 step4:ventilate_hashtags_in_portfolio Thu Sep 18 22:01:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 27012373 get user id for portfolio 27012373 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27012373 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','metal','pehd','carton','pet_clair','pet_fonce','background','environnement','mal_croppe','papier','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27012373 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','metal','pehd','carton','pet_clair','pet_fonce','background','environnement','mal_croppe','papier','flou')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27012373 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','metal','pehd','carton','pet_clair','pet_fonce','background','environnement','mal_croppe','papier','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27012428,27012429,27012430,27012431,27012432,27012433,27012434,27012435,27012436,27012437,27012438?tags=autre,metal,pehd,carton,pet_clair,pet_fonce,background,environnement,mal_croppe,papier,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1384932086, 1384931925, 1384931924, 1384931908, 1384931878, 1384931853, 1384931826, 1384931542, 1384931517, 1384931488, 1384931464, 1384931439, 1384931414] Looping around the photos to save general results len do output : 1 /27012373. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384932086', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931925', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931924', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931908', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931878', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931853', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931826', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931542', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931517', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931488', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931464', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931439', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931414', None, None, None, None, None, '3742863') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 14 time used for this insertion : 0.014420509338378906 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6350817680358887 time spend to save output : 0.014673233032226562 total time spend for step 4 : 1.6497550010681152 step5:final Thu Sep 18 22:01:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1384932086: ('0.20930166043447299',), 1384931925: ('0.20930166043447299',), 1384931924: ('0.20930166043447299',), 1384931908: ('0.20930166043447299',), 1384931878: ('0.20930166043447299',), 1384931853: ('0.20930166043447299',), 1384931826: ('0.20930166043447299',), 1384931542: ('0.20930166043447299',), 1384931517: ('0.20930166043447299',), 1384931488: ('0.20930166043447299',), 1384931464: ('0.20930166043447299',), 1384931439: ('0.20930166043447299',), 1384931414: ('0.20930166043447299',)} new output for save of step final : {1384932086: ('0.20930166043447299',), 1384931925: ('0.20930166043447299',), 1384931924: ('0.20930166043447299',), 1384931908: ('0.20930166043447299',), 1384931878: ('0.20930166043447299',), 1384931853: ('0.20930166043447299',), 1384931826: ('0.20930166043447299',), 1384931542: ('0.20930166043447299',), 1384931517: ('0.20930166043447299',), 1384931488: ('0.20930166043447299',), 1384931464: ('0.20930166043447299',), 1384931439: ('0.20930166043447299',), 1384931414: ('0.20930166043447299',)} [1384932086, 1384931925, 1384931924, 1384931908, 1384931878, 1384931853, 1384931826, 1384931542, 1384931517, 1384931488, 1384931464, 1384931439, 1384931414] Looping around the photos to save general results len do output : 13 /1384932086.Didn't retrieve data . /1384931925.Didn't retrieve data . /1384931924.Didn't retrieve data . /1384931908.Didn't retrieve data . /1384931878.Didn't retrieve data . /1384931853.Didn't retrieve data . /1384931826.Didn't retrieve data . /1384931542.Didn't retrieve data . /1384931517.Didn't retrieve data . /1384931488.Didn't retrieve data . /1384931464.Didn't retrieve data . /1384931439.Didn't retrieve data . /1384931414.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384932086', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931925', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931924', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931908', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931878', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931853', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931826', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931542', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931517', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931488', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931464', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931439', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931414', None, None, None, None, None, '3742863') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 39 time used for this insertion : 0.013418912887573242 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12828850746154785 time spend to save output : 0.014001846313476562 total time spend for step 5 : 0.14229035377502441 step6:blur_detection Thu Sep 18 22:01:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758225631_479308_1384932086_a09cbca9cd507bbab31744f163f3ab39.jpg resize: (1080, 1920) 1384932086 -2.6043551764852353 treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5.jpg resize: (1080, 1920) 1384931925 -2.7850235148753195 treat image : temp/1758225631_479308_1384931924_72d6042d2d17f43848aec0c1a7686f4d.jpg resize: (1080, 1920) 1384931924 -2.788420051571389 treat image : temp/1758225631_479308_1384931908_e32a22576c588cf2e589039e496ac16d.jpg resize: (1080, 1920) 1384931908 -2.6112138451295324 treat image : temp/1758225631_479308_1384931878_c195e61ee394285595909e2b95c051ad.jpg resize: (1080, 1920) 1384931878 -1.7596884289240882 treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0.jpg resize: (1080, 1920) 1384931853 -1.6280394529324387 treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312.jpg resize: (1080, 1920) 1384931826 -1.5896256534394573 treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9.jpg resize: (1080, 1920) 1384931542 -2.9096834185282567 treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783.jpg resize: (1080, 1920) 1384931517 -1.9113557869870905 treat image : temp/1758225631_479308_1384931488_73d01f73b8549f9799c35ece8d642c4e.jpg resize: (1080, 1920) 1384931488 -2.97345884641449 treat image : temp/1758225631_479308_1384931464_555bba8765e9824de025bf0c170da6b3.jpg resize: (1080, 1920) 1384931464 -0.9393170718768687 treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc.jpg resize: (1080, 1920) 1384931439 -2.2392575651779847 treat image : temp/1758225631_479308_1384931414_3cbc4dba18a2b953fce45f454cc5b52c.jpg resize: (1080, 1920) 1384931414 -1.8683450788274334 treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5_rle_crop_3966910940_0.png resize: (171, 99) 1384946170 -1.991550090713304 treat image : temp/1758225631_479308_1384931908_e32a22576c588cf2e589039e496ac16d_rle_crop_3966910945_0.png resize: (160, 101) 1384946171 -1.102188660102597 treat image : temp/1758225631_479308_1384931878_c195e61ee394285595909e2b95c051ad_rle_crop_3966910946_0.png resize: (154, 108) 1384946172 -0.9897001225991428 treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910948_0.png resize: (170, 106) 1384946173 -1.072753550974703 treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910949_0.png resize: (110, 138) 1384946174 -1.1440274641376877 treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910951_0.png resize: (90, 75) 1384946175 0.1740947076866678 treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910954_0.png resize: (41, 66) 1384946176 0.04538466305000539 treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910956_0.png resize: (162, 98) 1384946177 -1.5081631773502788 treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910955_0.png resize: (102, 134) 1384946178 -2.044761797456352 treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910962_0.png resize: (166, 100) 1384946179 -2.092503275274146 treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910960_0.png resize: (94, 97) 1384946181 -1.8237586555795304 treat image : temp/1758225631_479308_1384931488_73d01f73b8549f9799c35ece8d642c4e_rle_crop_3966910964_0.png resize: (113, 125) 1384946182 -1.7919440004172968 treat image : temp/1758225631_479308_1384931464_555bba8765e9824de025bf0c170da6b3_rle_crop_3966910965_0.png resize: (161, 106) 1384946183 -1.731298848789901 treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc_rle_crop_3966910969_0.png resize: (177, 103) 1384946184 -2.0730206276354206 treat image : temp/1758225631_479308_1384931414_3cbc4dba18a2b953fce45f454cc5b52c_rle_crop_3966910970_0.png resize: (185, 100) 1384946185 -1.6444240150058005 treat image : temp/1758225631_479308_1384931924_72d6042d2d17f43848aec0c1a7686f4d_rle_crop_3966910943_0.png resize: (107, 139) 1384946187 -1.4905444339130394 treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910947_0.png resize: (202, 103) 1384946188 -1.8550351121799333 treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910950_0.png resize: (105, 75) 1384946189 -0.180609574977188 treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910953_0.png resize: (118, 93) 1384946190 0.4990351265755539 treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5_rle_crop_3966910941_0.png resize: (133, 214) 1384946223 -3.4287505815765695 treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5_rle_crop_3966910942_0.png resize: (606, 373) 1384946224 0.09053076936329585 treat image : temp/1758225631_479308_1384931924_72d6042d2d17f43848aec0c1a7686f4d_rle_crop_3966910944_0.png resize: (214, 247) 1384946225 -1.4033616396485644 treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910952_0.png resize: (908, 1784) 1384946226 -1.9668857214908042 treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910957_0.png resize: (998, 1787) 1384946228 -2.7794958597761648 treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910959_0.png resize: (61, 155) 1384946229 -5.042269009007399 treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910958_0.png resize: (66, 205) 1384946230 -1.1932950586559075 treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910963_0.png resize: (925, 1831) 1384946232 -1.341404253979833 treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910961_0.png resize: (521, 328) 1384946233 0.022398617213401015 treat image : temp/1758225631_479308_1384931464_555bba8765e9824de025bf0c170da6b3_rle_crop_3966910966_0.png resize: (506, 354) 1384946234 0.2883142991943781 treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc_rle_crop_3966910967_0.png resize: (879, 993) 1384946235 -0.43633609634944565 treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc_rle_crop_3966910968_0.png resize: (491, 335) 1384946237 -0.548431064292216 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 44 time used for this insertion : 0.013319969177246094 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 44 time used for this insertion : 0.013641595840454102 save missing photos in datou_result : time spend for datou_step_exec : 12.387666702270508 time spend to save output : 0.032089948654174805 total time spend for step 6 : 12.419756650924683 step7:brightness Thu Sep 18 22:01:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758225631_479308_1384932086_a09cbca9cd507bbab31744f163f3ab39.jpg treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5.jpg treat image : temp/1758225631_479308_1384931924_72d6042d2d17f43848aec0c1a7686f4d.jpg treat image : temp/1758225631_479308_1384931908_e32a22576c588cf2e589039e496ac16d.jpg treat image : temp/1758225631_479308_1384931878_c195e61ee394285595909e2b95c051ad.jpg treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0.jpg treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312.jpg treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9.jpg treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783.jpg treat image : temp/1758225631_479308_1384931488_73d01f73b8549f9799c35ece8d642c4e.jpg treat image : temp/1758225631_479308_1384931464_555bba8765e9824de025bf0c170da6b3.jpg treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc.jpg treat image : temp/1758225631_479308_1384931414_3cbc4dba18a2b953fce45f454cc5b52c.jpg treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5_rle_crop_3966910940_0.png treat image : temp/1758225631_479308_1384931908_e32a22576c588cf2e589039e496ac16d_rle_crop_3966910945_0.png treat image : temp/1758225631_479308_1384931878_c195e61ee394285595909e2b95c051ad_rle_crop_3966910946_0.png treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910948_0.png treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910949_0.png treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910951_0.png treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910954_0.png treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910956_0.png treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910955_0.png treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910962_0.png treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910960_0.png treat image : temp/1758225631_479308_1384931488_73d01f73b8549f9799c35ece8d642c4e_rle_crop_3966910964_0.png treat image : temp/1758225631_479308_1384931464_555bba8765e9824de025bf0c170da6b3_rle_crop_3966910965_0.png treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc_rle_crop_3966910969_0.png treat image : temp/1758225631_479308_1384931414_3cbc4dba18a2b953fce45f454cc5b52c_rle_crop_3966910970_0.png treat image : temp/1758225631_479308_1384931924_72d6042d2d17f43848aec0c1a7686f4d_rle_crop_3966910943_0.png treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910947_0.png treat image : temp/1758225631_479308_1384931853_3257fccb33ef4911cf24098d4f1961f0_rle_crop_3966910950_0.png treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910953_0.png treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5_rle_crop_3966910941_0.png treat image : temp/1758225631_479308_1384931925_656eac7bc343a39506c0131432f410f5_rle_crop_3966910942_0.png treat image : temp/1758225631_479308_1384931924_72d6042d2d17f43848aec0c1a7686f4d_rle_crop_3966910944_0.png treat image : temp/1758225631_479308_1384931826_67cf60820a3d6e46d41d7d606a71b312_rle_crop_3966910952_0.png treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910957_0.png treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910959_0.png treat image : temp/1758225631_479308_1384931542_ce2125d6c3822e74b01d273346f566c9_rle_crop_3966910958_0.png treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910963_0.png treat image : temp/1758225631_479308_1384931517_7e66e6195a9012905991fc662b68b783_rle_crop_3966910961_0.png treat image : temp/1758225631_479308_1384931464_555bba8765e9824de025bf0c170da6b3_rle_crop_3966910966_0.png treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc_rle_crop_3966910967_0.png treat image : temp/1758225631_479308_1384931439_ae4c4c1a51a85c55746731080eb45bdc_rle_crop_3966910968_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 44 time used for this insertion : 0.013657331466674805 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 44 time used for this insertion : 0.012935161590576172 save missing photos in datou_result : time spend for datou_step_exec : 3.757446765899658 time spend to save output : 0.0318300724029541 total time spend for step 7 : 3.7892768383026123 step8:velours_tree Thu Sep 18 22:02:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.20729279518127441 time spend to save output : 3.4809112548828125e-05 total time spend for step 8 : 0.20732760429382324 step9:send_mail_cod Thu Sep 18 22:02:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P27012373_18-09-2025_22_02_00.pdf 27012428 imagette270124281758225720 27012429 imagette270124291758225720 27012430 imagette270124301758225720 27012431 change filename to text .change filename to text .change filename to text .change filename to text .imagette270124311758225720 27012432 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette270124321758225721 27012433 imagette270124331758225722 27012434 imagette270124341758225722 27012436 imagette270124361758225722 27012437 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette270124371758225722 27012438 imagette270124381758225723 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27012373 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27012428,27012429,27012430,27012431,27012432,27012433,27012434,27012435,27012436,27012437,27012438?tags=autre,metal,pehd,carton,pet_clair,pet_fonce,background,environnement,mal_croppe,papier,flou args[1384932086] : ((1384932086, -2.6043551764852353, 492609224), (1384932086, 0.5500088374518707, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931925] : ((1384931925, -2.7850235148753195, 492609224), (1384931925, 0.6122787929805527, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931924] : ((1384931924, -2.788420051571389, 492609224), (1384931924, 0.5610892510297215, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931908] : ((1384931908, -2.6112138451295324, 492609224), (1384931908, 0.6472072360019808, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931878] : ((1384931878, -1.7596884289240882, 492688767), (1384931878, 0.5256154479579616, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931853] : ((1384931853, -1.6280394529324387, 492688767), (1384931853, 0.40351691438679865, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931826] : ((1384931826, -1.5896256534394573, 492688767), (1384931826, 0.5497070835267791, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931542] : ((1384931542, -2.9096834185282567, 492609224), (1384931542, 0.4128314930965791, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931517] : ((1384931517, -1.9113557869870905, 492688767), (1384931517, 0.34276954137362287, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931488] : ((1384931488, -2.97345884641449, 492609224), (1384931488, 0.4299076983583379, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931464] : ((1384931464, -0.9393170718768687, 492688767), (1384931464, 0.5978504837508662, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931439] : ((1384931439, -2.2392575651779847, 492609224), (1384931439, 0.5037658236958067, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com args[1384931414] : ((1384931414, -1.8683450788274334, 492688767), (1384931414, 0.608089183448193, 2107752395), '0.20930166043447299') We are sending mail with results at report@fotonower.com refus_total : 0.20930166043447299 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27012373 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27012373_18-09-2025_22_02_00.pdf results_Auto_P27012373_18-09-2025_22_02_00.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27012373_18-09-2025_22_02_00.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','27012373','results_Auto_P27012373_18-09-2025_22_02_00.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27012373_18-09-2025_22_02_00.pdf','pdf','','0.69','0.20930166043447299') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/27012373

https://www.fotonower.com/image?json=false&list_photos_id=1384932086
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931925
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931924
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931908
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931878
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931853
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931826
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931542
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931517
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931488
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931464
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931439
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384931414
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 20.93%
Veuillez trouver les photos des contaminants.

exemples de contaminants: carton: https://www.fotonower.com/view/27012431?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/27012432?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/27012437?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27012373_18-09-2025_22_02_00.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/27012428,27012429,27012430,27012431,27012432,27012433,27012434,27012435,27012436,27012437,27012438?tags=autre,metal,pehd,carton,pet_clair,pet_fonce,background,environnement,mal_croppe,papier,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 18 Sep 2025 20:02:06 GMT Content-Length: 0 Connection: close X-Message-Id: C0CHDFteScWjl-LZiBtDdA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1384932086, 1384931925, 1384931924, 1384931908, 1384931878, 1384931853, 1384931826, 1384931542, 1384931517, 1384931488, 1384931464, 1384931439, 1384931414] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384932086', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931925', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931924', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931908', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931878', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931853', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931826', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931542', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931517', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931488', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931464', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931439', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931414', None, None, None, None, None, '3742863') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.013264179229736328 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.185828447341919 time spend to save output : 0.013546466827392578 total time spend for step 9 : 5.1993749141693115 step10:split_time_score Thu Sep 18 22:02:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 13),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 18092025 27012373 Nombre de photos uploadées : 13 / 23040 (0%) 18092025 27012373 Nombre de photos taguées (types de déchets): 0 / 13 (0%) 18092025 27012373 Nombre de photos taguées (volume) : 0 / 13 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0006480216979980469 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.1990034580230713 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.029968609708193043 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26993632_18-09-2025_09_01_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26993632 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26993632 AND mptpi.`type`=3594 To do Qualite : 0.07461375911896746 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26994368_18-09-2025_09_32_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26994368 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26994368 AND mptpi.`type`=3594 To do Qualite : 0.13248333547668037 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26994376_18-09-2025_09_21_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26994376 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26994376 AND mptpi.`type`=3594 To do Qualite : 0.02829922027290448 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26994377_18-09-2025_09_11_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26994377 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26994377 AND mptpi.`type`=3594 To do Qualite : 0.22073720421810702 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26998416_18-09-2025_13_02_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26998416 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26998416 AND mptpi.`type`=3594 To do Qualite : 0.06378718171296298 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26997919_18-09-2025_12_41_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26997919 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26997919 AND mptpi.`type`=3594 To do Qualite : 0.10073354713220166 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26996058_18-09-2025_13_11_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26996058 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26996058 AND mptpi.`type`=3594 To do Qualite : 0.14480436599794244 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26997922_18-09-2025_12_31_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26997922 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26997922 AND mptpi.`type`=3594 To do Qualite : 0.10946210026577505 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26999691_18-09-2025_14_22_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26999691 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26999691 AND mptpi.`type`=3594 To do Qualite : 0.05668700569058639 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27000677_18-09-2025_16_36_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27000677 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27000677 AND mptpi.`type`=3594 To do Qualite : 0.029082593878600828 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27007399_18-09-2025_20_21_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27007399 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27007399 AND mptpi.`type`=3594 To do Qualite : 0.06740903964862299 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27007401_18-09-2025_20_12_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27007401 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27007401 AND mptpi.`type`=3594 To do Qualite : 0.08279116957502375 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27007402_18-09-2025_20_01_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27007402 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27007402 AND mptpi.`type`=3594 To do Qualite : 0.03057369569088319 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27002076_18-09-2025_17_11_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27002076 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27002076 AND mptpi.`type`=3594 To do Qualite : 0.05614663708847737 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27003818_18-09-2025_18_41_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27003818 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27003818 AND mptpi.`type`=3594 To do Qualite : 0.20930166043447299 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27012373_18-09-2025_22_02_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27012373 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27012373 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'18092025': {'nb_upload': 13, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1384932086, 1384931925, 1384931924, 1384931908, 1384931878, 1384931853, 1384931826, 1384931542, 1384931517, 1384931488, 1384931464, 1384931439, 1384931414] Looping around the photos to save general results len do output : 1 /27012373Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384932086', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931925', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931924', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931908', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931878', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931853', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931826', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931542', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931517', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931488', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931464', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931439', None, None, None, None, None, '3742863') ('3318', None, None, None, None, None, None, None, '3742863') ('3318', '27012373', '1384931414', None, None, None, None, None, '3742863') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 14 time used for this insertion : 0.01356816291809082 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.2933573722839355 time spend to save output : 0.013850688934326172 total time spend for step 10 : 1.3072080612182617 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 13 set_done_treatment 60.24user 22.84system 1:41.01elapsed 82%CPU (0avgtext+0avgdata 2794284maxresident)k 567816inputs+49344outputs (605major+1882846minor)pagefaults 0swaps