python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3172014 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3660766'] with mtr_portfolio_ids : ['26504438'] and first list_photo_ids : [] new path : /proc/3172014/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.5246522426605225 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed Sep 3 21:40:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10601 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-03 21:40:34.229230: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-03 21:40:34.256682: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-03 21:40:34.259798: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe594000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-03 21:40:34.259865: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-03 21:40:34.264391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-03 21:40:34.415110: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x30788d90 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-03 21:40:34.415189: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-03 21:40:34.416548: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-03 21:40:34.417111: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-03 21:40:34.421040: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-03 21:40:34.424718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-03 21:40:34.425363: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-03 21:40:34.428021: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-03 21:40:34.429364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-03 21:40:34.434959: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-03 21:40:34.436574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-03 21:40:34.436666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-03 21:40:34.437534: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-03 21:40:34.437561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-03 21:40:34.437573: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-03 21:40:34.439399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9675 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-03 21:40:34.817477: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-03 21:40:34.817602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-03 21:40:34.817625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-03 21:40:34.817645: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-03 21:40:34.817662: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-03 21:40:34.817680: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-03 21:40:34.817698: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-03 21:40:34.817717: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-03 21:40:34.819266: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-03 21:40:34.820854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-03 21:40:34.820905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-03 21:40:34.820924: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-03 21:40:34.820941: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-03 21:40:34.820959: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-03 21:40:34.820975: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-03 21:40:34.820993: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-03 21:40:34.821009: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-03 21:40:34.822250: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-03 21:40:34.822286: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-03 21:40:34.822294: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-03 21:40:34.822301: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-03 21:40:34.823604: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9675 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-03 21:40:44.766669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-03 21:40:44.961041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (2160, 3840, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3840.00000 nb d'objets trouves : 23 Detection mask done ! Trying to reset tf kernel 3172610 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1532 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6821 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0004413127899169922 nb_pixel_total : 12501 time to create 1 rle with old method : 0.013843536376953125 length of segment : 207 time for calcul the mask position with numpy : 0.0005922317504882812 nb_pixel_total : 25427 time to create 1 rle with old method : 0.02797722816467285 length of segment : 225 time for calcul the mask position with numpy : 0.0010857582092285156 nb_pixel_total : 59456 time to create 1 rle with old method : 0.06448507308959961 length of segment : 234 time for calcul the mask position with numpy : 0.0013813972473144531 nb_pixel_total : 76772 time to create 1 rle with old method : 0.08669543266296387 length of segment : 354 time for calcul the mask position with numpy : 0.00042057037353515625 nb_pixel_total : 22523 time to create 1 rle with old method : 0.025180339813232422 length of segment : 167 time for calcul the mask position with numpy : 0.0035295486450195312 nb_pixel_total : 144619 time to create 1 rle with old method : 0.15928053855895996 length of segment : 428 time for calcul the mask position with numpy : 0.0004165172576904297 nb_pixel_total : 20785 time to create 1 rle with old method : 0.022312641143798828 length of segment : 163 time for calcul the mask position with numpy : 0.0007026195526123047 nb_pixel_total : 49174 time to create 1 rle with old method : 0.05468320846557617 length of segment : 320 time for calcul the mask position with numpy : 0.0020940303802490234 nb_pixel_total : 100439 time to create 1 rle with old method : 0.10960650444030762 length of segment : 380 time for calcul the mask position with numpy : 0.0009665489196777344 nb_pixel_total : 56021 time to create 1 rle with old method : 0.060463905334472656 length of segment : 300 time for calcul the mask position with numpy : 0.0003380775451660156 nb_pixel_total : 21293 time to create 1 rle with old method : 0.02381157875061035 length of segment : 205 time for calcul the mask position with numpy : 0.00037288665771484375 nb_pixel_total : 27028 time to create 1 rle with old method : 0.029443740844726562 length of segment : 181 time for calcul the mask position with numpy : 0.0007989406585693359 nb_pixel_total : 49526 time to create 1 rle with old method : 0.05453228950500488 length of segment : 272 time for calcul the mask position with numpy : 0.0001685619354248047 nb_pixel_total : 10014 time to create 1 rle with old method : 0.01126241683959961 length of segment : 65 time for calcul the mask position with numpy : 0.0011134147644042969 nb_pixel_total : 74798 time to create 1 rle with old method : 0.08208942413330078 length of segment : 300 time for calcul the mask position with numpy : 0.00041794776916503906 nb_pixel_total : 14128 time to create 1 rle with old method : 0.015950441360473633 length of segment : 313 time for calcul the mask position with numpy : 0.0002040863037109375 nb_pixel_total : 10075 time to create 1 rle with old method : 0.01129913330078125 length of segment : 123 time for calcul the mask position with numpy : 0.0002453327178955078 nb_pixel_total : 12197 time to create 1 rle with old method : 0.013940095901489258 length of segment : 155 time for calcul the mask position with numpy : 0.0001499652862548828 nb_pixel_total : 6983 time to create 1 rle with old method : 0.00756382942199707 length of segment : 136 time for calcul the mask position with numpy : 0.0008437633514404297 nb_pixel_total : 62005 time to create 1 rle with old method : 0.07032275199890137 length of segment : 350 time for calcul the mask position with numpy : 0.0005407333374023438 nb_pixel_total : 27566 time to create 1 rle with old method : 0.030200958251953125 length of segment : 193 time for calcul the mask position with numpy : 0.00024628639221191406 nb_pixel_total : 10426 time to create 1 rle with old method : 0.011907339096069336 length of segment : 110 time for calcul the mask position with numpy : 0.0005719661712646484 nb_pixel_total : 38704 time to create 1 rle with old method : 0.042424917221069336 length of segment : 199 time for calcul the mask position with numpy : 0.00045871734619140625 nb_pixel_total : 32950 time to create 1 rle with old method : 0.03843998908996582 length of segment : 282 time for calcul the mask position with numpy : 0.0005662441253662109 nb_pixel_total : 33688 time to create 1 rle with old method : 0.03717923164367676 length of segment : 281 time for calcul the mask position with numpy : 0.00020837783813476562 nb_pixel_total : 11987 time to create 1 rle with old method : 0.013531923294067383 length of segment : 113 time for calcul the mask position with numpy : 0.00047278404235839844 nb_pixel_total : 17304 time to create 1 rle with old method : 0.021015167236328125 length of segment : 404 time for calcul the mask position with numpy : 0.0019028186798095703 nb_pixel_total : 62470 time to create 1 rle with old method : 0.068817138671875 length of segment : 604 time for calcul the mask position with numpy : 0.0004291534423828125 nb_pixel_total : 23529 time to create 1 rle with old method : 0.025977373123168945 length of segment : 182 time for calcul the mask position with numpy : 0.00032067298889160156 nb_pixel_total : 10334 time to create 1 rle with old method : 0.011662960052490234 length of segment : 101 time for calcul the mask position with numpy : 0.00037860870361328125 nb_pixel_total : 25421 time to create 1 rle with old method : 0.02897357940673828 length of segment : 149 time for calcul the mask position with numpy : 0.0006363391876220703 nb_pixel_total : 34928 time to create 1 rle with old method : 0.03968095779418945 length of segment : 293 time for calcul the mask position with numpy : 0.0007665157318115234 nb_pixel_total : 26151 time to create 1 rle with old method : 0.02995920181274414 length of segment : 290 time for calcul the mask position with numpy : 0.0002448558807373047 nb_pixel_total : 6114 time to create 1 rle with old method : 0.006997823715209961 length of segment : 161 time for calcul the mask position with numpy : 0.0028438568115234375 nb_pixel_total : 51753 time to create 1 rle with old method : 0.05564236640930176 length of segment : 579 time for calcul the mask position with numpy : 0.0027582645416259766 nb_pixel_total : 98213 time to create 1 rle with old method : 0.10463809967041016 length of segment : 398 time for calcul the mask position with numpy : 0.0008380413055419922 nb_pixel_total : 22782 time to create 1 rle with old method : 0.024636507034301758 length of segment : 190 time for calcul the mask position with numpy : 0.0006859302520751953 nb_pixel_total : 15487 time to create 1 rle with old method : 0.017450571060180664 length of segment : 150 time for calcul the mask position with numpy : 0.0011467933654785156 nb_pixel_total : 9645 time to create 1 rle with old method : 0.010852813720703125 length of segment : 368 time for calcul the mask position with numpy : 0.0007724761962890625 nb_pixel_total : 24829 time to create 1 rle with old method : 0.027441740036010742 length of segment : 198 time for calcul the mask position with numpy : 0.0008258819580078125 nb_pixel_total : 21202 time to create 1 rle with old method : 0.02275705337524414 length of segment : 260 time for calcul the mask position with numpy : 0.00025463104248046875 nb_pixel_total : 8697 time to create 1 rle with old method : 0.009503364562988281 length of segment : 82 time for calcul the mask position with numpy : 0.0006921291351318359 nb_pixel_total : 19052 time to create 1 rle with old method : 0.02165079116821289 length of segment : 171 time for calcul the mask position with numpy : 0.0008034706115722656 nb_pixel_total : 7130 time to create 1 rle with old method : 0.008035421371459961 length of segment : 220 time for calcul the mask position with numpy : 0.00043463706970214844 nb_pixel_total : 11270 time to create 1 rle with old method : 0.013151407241821289 length of segment : 127 time for calcul the mask position with numpy : 0.0008182525634765625 nb_pixel_total : 34046 time to create 1 rle with old method : 0.03772163391113281 length of segment : 197 time for calcul the mask position with numpy : 0.001373291015625 nb_pixel_total : 46505 time to create 1 rle with old method : 0.05098128318786621 length of segment : 321 time for calcul the mask position with numpy : 0.0007920265197753906 nb_pixel_total : 27039 time to create 1 rle with old method : 0.02867579460144043 length of segment : 246 time for calcul the mask position with numpy : 0.0008785724639892578 nb_pixel_total : 23942 time to create 1 rle with old method : 0.026623964309692383 length of segment : 222 time for calcul the mask position with numpy : 0.0014042854309082031 nb_pixel_total : 54063 time to create 1 rle with old method : 0.05832076072692871 length of segment : 278 time for calcul the mask position with numpy : 0.0017399787902832031 nb_pixel_total : 70233 time to create 1 rle with old method : 0.0749516487121582 length of segment : 302 time for calcul the mask position with numpy : 0.001215219497680664 nb_pixel_total : 43244 time to create 1 rle with old method : 0.05564737319946289 length of segment : 313 time for calcul the mask position with numpy : 0.001016378402709961 nb_pixel_total : 33896 time to create 1 rle with old method : 0.04536104202270508 length of segment : 220 time for calcul the mask position with numpy : 0.0002205371856689453 nb_pixel_total : 11403 time to create 1 rle with old method : 0.012760639190673828 length of segment : 110 time for calcul the mask position with numpy : 0.00046896934509277344 nb_pixel_total : 20226 time to create 1 rle with old method : 0.02154374122619629 length of segment : 142 time for calcul the mask position with numpy : 0.0020902156829833984 nb_pixel_total : 86291 time to create 1 rle with old method : 0.09308719635009766 length of segment : 474 time for calcul the mask position with numpy : 0.0008037090301513672 nb_pixel_total : 41163 time to create 1 rle with old method : 0.04555988311767578 length of segment : 112 time for calcul the mask position with numpy : 0.0006268024444580078 nb_pixel_total : 27150 time to create 1 rle with old method : 0.02976059913635254 length of segment : 244 time for calcul the mask position with numpy : 0.002061128616333008 nb_pixel_total : 87277 time to create 1 rle with old method : 0.09773683547973633 length of segment : 358 time for calcul the mask position with numpy : 0.0014166831970214844 nb_pixel_total : 54820 time to create 1 rle with old method : 0.06015801429748535 length of segment : 300 time for calcul the mask position with numpy : 0.0008165836334228516 nb_pixel_total : 28140 time to create 1 rle with old method : 0.02949047088623047 length of segment : 235 time for calcul the mask position with numpy : 0.0008499622344970703 nb_pixel_total : 35574 time to create 1 rle with old method : 0.03812766075134277 length of segment : 342 time for calcul the mask position with numpy : 0.0004150867462158203 nb_pixel_total : 9447 time to create 1 rle with old method : 0.011297225952148438 length of segment : 99 time for calcul the mask position with numpy : 0.001588582992553711 nb_pixel_total : 38021 time to create 1 rle with old method : 0.040467023849487305 length of segment : 303 time for calcul the mask position with numpy : 0.0015811920166015625 nb_pixel_total : 53165 time to create 1 rle with old method : 0.05529618263244629 length of segment : 326 time for calcul the mask position with numpy : 0.0009930133819580078 nb_pixel_total : 26947 time to create 1 rle with old method : 0.03453850746154785 length of segment : 234 time for calcul the mask position with numpy : 0.001855611801147461 nb_pixel_total : 58749 time to create 1 rle with old method : 0.06411862373352051 length of segment : 362 time for calcul the mask position with numpy : 0.0008275508880615234 nb_pixel_total : 23410 time to create 1 rle with old method : 0.02539682388305664 length of segment : 187 time for calcul the mask position with numpy : 0.0008263587951660156 nb_pixel_total : 34405 time to create 1 rle with old method : 0.03756904602050781 length of segment : 338 time for calcul the mask position with numpy : 0.00966191291809082 nb_pixel_total : 201058 time to create 1 rle with new method : 0.02244710922241211 length of segment : 527 time for calcul the mask position with numpy : 0.0015273094177246094 nb_pixel_total : 39924 time to create 1 rle with old method : 0.04302549362182617 length of segment : 327 time for calcul the mask position with numpy : 0.0014843940734863281 nb_pixel_total : 37573 time to create 1 rle with old method : 0.04328203201293945 length of segment : 196 time for calcul the mask position with numpy : 0.0014903545379638672 nb_pixel_total : 54428 time to create 1 rle with old method : 0.061215877532958984 length of segment : 498 time for calcul the mask position with numpy : 0.0009176731109619141 nb_pixel_total : 28716 time to create 1 rle with old method : 0.033075571060180664 length of segment : 326 time for calcul the mask position with numpy : 0.0007424354553222656 nb_pixel_total : 18238 time to create 1 rle with old method : 0.020136594772338867 length of segment : 178 time for calcul the mask position with numpy : 0.0010986328125 nb_pixel_total : 34331 time to create 1 rle with old method : 0.037168264389038086 length of segment : 290 time for calcul the mask position with numpy : 0.0021276473999023438 nb_pixel_total : 56570 time to create 1 rle with old method : 0.06087470054626465 length of segment : 385 time for calcul the mask position with numpy : 0.0004916191101074219 nb_pixel_total : 23084 time to create 1 rle with old method : 0.025582075119018555 length of segment : 191 time spent for convertir_results : 6.1714372634887695 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 78 chid ids of type : 3594 Number RLEs to save : 20166 save missing photos in datou_result : time spend for datou_step_exec : 49.58536672592163 time spend to save output : 1.3440611362457275 total time spend for step 1 : 50.92942786216736 step2:crop_condition Wed Sep 3 21:41:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 78 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 60 About to insert : list_path_to_insert length 60 new photo from crops ! About to upload 60 photos upload in portfolio : 3736932 init cache_photo without model_param we have 60 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756928496_3172014 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 60 photos in the portfolio 3736932 time of upload the photos Elapsed time : 18.745113611221313 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3736932 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756928517_3172014 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 8 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.1619460582733154 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3736932 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756928521_3172014 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.579113245010376 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756928525_3172014 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5522019863128662 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1381228491, 1381228489, 1381228477, 1381228426, 1381228415, 1381228412, 1381228403, 1381228399] Looping around the photos to save general results len do output : 78 /1381249070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249135Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249141Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381249161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228491', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228489', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228477', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228426', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228415', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228412', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228403', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228399', None, None, None, None, None, '3660766') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 242 time used for this insertion : 0.026586055755615234 save_final save missing photos in datou_result : time spend for datou_step_exec : 43.1502320766449 time spend to save output : 0.02960801124572754 total time spend for step 2 : 43.179840087890625 step3:rle_unique_nms_with_priority Wed Sep 3 21:42:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 78 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 3.571077585220337 time for calcul the mask position with numpy : 0.5545580387115479 nb_pixel_total : 7782837 time to create 1 rle with new method : 0.8070576190948486 time for calcul the mask position with numpy : 0.023264169692993164 nb_pixel_total : 100439 time to create 1 rle with old method : 0.11044549942016602 time for calcul the mask position with numpy : 0.023471355438232422 nb_pixel_total : 49174 time to create 1 rle with old method : 0.051041603088378906 time for calcul the mask position with numpy : 0.023603200912475586 nb_pixel_total : 20785 time to create 1 rle with old method : 0.021422147750854492 time for calcul the mask position with numpy : 0.02392745018005371 nb_pixel_total : 144486 time to create 1 rle with old method : 0.1513993740081787 time for calcul the mask position with numpy : 0.025308847427368164 nb_pixel_total : 22523 time to create 1 rle with old method : 0.02462458610534668 time for calcul the mask position with numpy : 0.02433490753173828 nb_pixel_total : 76772 time to create 1 rle with old method : 0.07922840118408203 time for calcul the mask position with numpy : 0.02466559410095215 nb_pixel_total : 59456 time to create 1 rle with old method : 0.06585574150085449 time for calcul the mask position with numpy : 0.024356365203857422 nb_pixel_total : 25427 time to create 1 rle with old method : 0.027889251708984375 time for calcul the mask position with numpy : 0.03064703941345215 nb_pixel_total : 12501 time to create 1 rle with old method : 0.013327598571777344 create new chi : 2.1641147136688232 time to delete rle : 0.025860071182250977 batch 1 Loaded 19 chid ids of type : 3594 +++++++++Number RLEs to save : 7104 TO DO : save crop sub photo not yet done ! save time : 0.4628303050994873 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 3.634720802307129 time for calcul the mask position with numpy : 0.5748534202575684 nb_pixel_total : 7950334 time to create 1 rle with new method : 0.7477529048919678 time for calcul the mask position with numpy : 0.02350139617919922 nb_pixel_total : 62005 time to create 1 rle with old method : 0.0643930435180664 time for calcul the mask position with numpy : 0.022583961486816406 nb_pixel_total : 6981 time to create 1 rle with old method : 0.007140159606933594 time for calcul the mask position with numpy : 0.023177385330200195 nb_pixel_total : 12197 time to create 1 rle with old method : 0.012608528137207031 time for calcul the mask position with numpy : 0.022500038146972656 nb_pixel_total : 10075 time to create 1 rle with old method : 0.010320186614990234 time for calcul the mask position with numpy : 0.022982358932495117 nb_pixel_total : 14128 time to create 1 rle with old method : 0.014548301696777344 time for calcul the mask position with numpy : 0.023690462112426758 nb_pixel_total : 74798 time to create 1 rle with old method : 0.07656311988830566 time for calcul the mask position with numpy : 0.023288726806640625 nb_pixel_total : 10014 time to create 1 rle with old method : 0.010339498519897461 time for calcul the mask position with numpy : 0.023435592651367188 nb_pixel_total : 49526 time to create 1 rle with old method : 0.05096030235290527 time for calcul the mask position with numpy : 0.02421855926513672 nb_pixel_total : 27028 time to create 1 rle with old method : 0.0282900333404541 time for calcul the mask position with numpy : 0.023194551467895508 nb_pixel_total : 21293 time to create 1 rle with old method : 0.0224764347076416 time for calcul the mask position with numpy : 0.024485111236572266 nb_pixel_total : 56021 time to create 1 rle with old method : 0.05920052528381348 create new chi : 1.9768848419189453 time to delete rle : 0.0009353160858154297 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++Number RLEs to save : 6947 TO DO : save crop sub photo not yet done ! save time : 0.44135594367980957 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 3.125028133392334 time for calcul the mask position with numpy : 0.4373936653137207 nb_pixel_total : 8035776 time to create 1 rle with new method : 0.7133376598358154 time for calcul the mask position with numpy : 0.024419784545898438 nb_pixel_total : 23529 time to create 1 rle with old method : 0.025755643844604492 time for calcul the mask position with numpy : 0.0230863094329834 nb_pixel_total : 62470 time to create 1 rle with old method : 0.06599545478820801 time for calcul the mask position with numpy : 0.023066043853759766 nb_pixel_total : 17304 time to create 1 rle with old method : 0.01828765869140625 time for calcul the mask position with numpy : 0.022705793380737305 nb_pixel_total : 11987 time to create 1 rle with old method : 0.012627840042114258 time for calcul the mask position with numpy : 0.023351192474365234 nb_pixel_total : 33688 time to create 1 rle with old method : 0.034980058670043945 time for calcul the mask position with numpy : 0.02368760108947754 nb_pixel_total : 32950 time to create 1 rle with old method : 0.03477835655212402 time for calcul the mask position with numpy : 0.024281978607177734 nb_pixel_total : 38704 time to create 1 rle with old method : 0.04335904121398926 time for calcul the mask position with numpy : 0.024754047393798828 nb_pixel_total : 10426 time to create 1 rle with old method : 0.011668205261230469 time for calcul the mask position with numpy : 0.024246931076049805 nb_pixel_total : 27566 time to create 1 rle with old method : 0.030962705612182617 create new chi : 1.6823770999908447 time to delete rle : 0.0009732246398925781 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++Number RLEs to save : 6896 TO DO : save crop sub photo not yet done ! save time : 0.4427015781402588 nb_obj : 11 nb_hashtags : 2 time to prepare the origin masks : 4.731554746627808 time for calcul the mask position with numpy : 0.6683487892150879 nb_pixel_total : 7972277 time to create 1 rle with new method : 0.6290688514709473 time for calcul the mask position with numpy : 0.0258486270904541 nb_pixel_total : 24829 time to create 1 rle with old method : 0.02901482582092285 time for calcul the mask position with numpy : 0.026192188262939453 nb_pixel_total : 6111 time to create 1 rle with old method : 0.0068094730377197266 time for calcul the mask position with numpy : 0.024445772171020508 nb_pixel_total : 15487 time to create 1 rle with old method : 0.01680588722229004 time for calcul the mask position with numpy : 0.024321317672729492 nb_pixel_total : 22782 time to create 1 rle with old method : 0.02543330192565918 time for calcul the mask position with numpy : 0.027006149291992188 nb_pixel_total : 98213 time to create 1 rle with old method : 0.1071310043334961 time for calcul the mask position with numpy : 0.024560213088989258 nb_pixel_total : 51753 time to create 1 rle with old method : 0.05570697784423828 time for calcul the mask position with numpy : 0.024553298950195312 nb_pixel_total : 6114 time to create 1 rle with old method : 0.006827592849731445 time for calcul the mask position with numpy : 0.02539801597595215 nb_pixel_total : 26151 time to create 1 rle with old method : 0.028927326202392578 time for calcul the mask position with numpy : 0.025748729705810547 nb_pixel_total : 34928 time to create 1 rle with old method : 0.03720545768737793 time for calcul the mask position with numpy : 0.02569437026977539 nb_pixel_total : 25421 time to create 1 rle with old method : 0.029623985290527344 time for calcul the mask position with numpy : 0.02402186393737793 nb_pixel_total : 10334 time to create 1 rle with old method : 0.01096796989440918 create new chi : 1.9755070209503174 time to delete rle : 0.0010418891906738281 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 7781 TO DO : save crop sub photo not yet done ! save time : 0.5060591697692871 nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 3.1227800846099854 time for calcul the mask position with numpy : 0.5655157566070557 nb_pixel_total : 8119459 time to create 1 rle with new method : 0.6726393699645996 time for calcul the mask position with numpy : 0.023294687271118164 nb_pixel_total : 27039 time to create 1 rle with old method : 0.028193235397338867 time for calcul the mask position with numpy : 0.02338409423828125 nb_pixel_total : 46505 time to create 1 rle with old method : 0.05002236366271973 time for calcul the mask position with numpy : 0.024544477462768555 nb_pixel_total : 34046 time to create 1 rle with old method : 0.036272525787353516 time for calcul the mask position with numpy : 0.02371382713317871 nb_pixel_total : 11270 time to create 1 rle with old method : 0.012127876281738281 time for calcul the mask position with numpy : 0.023782014846801758 nb_pixel_total : 7130 time to create 1 rle with old method : 0.011259317398071289 time for calcul the mask position with numpy : 0.037946224212646484 nb_pixel_total : 19052 time to create 1 rle with old method : 0.020057201385498047 time for calcul the mask position with numpy : 0.03654956817626953 nb_pixel_total : 8697 time to create 1 rle with old method : 0.009187936782836914 time for calcul the mask position with numpy : 0.03690505027770996 nb_pixel_total : 21202 time to create 1 rle with old method : 0.02204608917236328 create new chi : 1.6987743377685547 time to delete rle : 0.0008611679077148438 batch 1 Loaded 17 chid ids of type : 3594 +++++++++++Number RLEs to save : 5408 TO DO : save crop sub photo not yet done ! save time : 0.3458068370819092 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 3.2653396129608154 time for calcul the mask position with numpy : 0.49663615226745605 nb_pixel_total : 7883181 time to create 1 rle with new method : 0.9140496253967285 time for calcul the mask position with numpy : 0.023914098739624023 nb_pixel_total : 27150 time to create 1 rle with old method : 0.02845931053161621 time for calcul the mask position with numpy : 0.024204015731811523 nb_pixel_total : 41163 time to create 1 rle with old method : 0.04456305503845215 time for calcul the mask position with numpy : 0.024588823318481445 nb_pixel_total : 85899 time to create 1 rle with old method : 0.09139418601989746 time for calcul the mask position with numpy : 0.02302241325378418 nb_pixel_total : 20226 time to create 1 rle with old method : 0.02114105224609375 time for calcul the mask position with numpy : 0.023669719696044922 nb_pixel_total : 11403 time to create 1 rle with old method : 0.01191091537475586 time for calcul the mask position with numpy : 0.024506807327270508 nb_pixel_total : 33896 time to create 1 rle with old method : 0.03966665267944336 time for calcul the mask position with numpy : 0.026338815689086914 nb_pixel_total : 43244 time to create 1 rle with old method : 0.046133995056152344 time for calcul the mask position with numpy : 0.0240633487701416 nb_pixel_total : 70233 time to create 1 rle with old method : 0.07388567924499512 time for calcul the mask position with numpy : 0.02385568618774414 nb_pixel_total : 54063 time to create 1 rle with old method : 0.05855607986450195 time for calcul the mask position with numpy : 0.03950357437133789 nb_pixel_total : 23942 time to create 1 rle with old method : 0.02688145637512207 create new chi : 2.1552388668060303 time to delete rle : 0.0011942386627197266 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++Number RLEs to save : 6934 TO DO : save crop sub photo not yet done ! save time : 0.4523453712463379 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 2.5814735889434814 time for calcul the mask position with numpy : 0.4087367057800293 nb_pixel_total : 8088589 time to create 1 rle with new method : 0.7683916091918945 time for calcul the mask position with numpy : 0.02834773063659668 nb_pixel_total : 35574 time to create 1 rle with old method : 0.03726625442504883 time for calcul the mask position with numpy : 0.024087905883789062 nb_pixel_total : 28140 time to create 1 rle with old method : 0.02984929084777832 time for calcul the mask position with numpy : 0.02422022819519043 nb_pixel_total : 54820 time to create 1 rle with old method : 0.05772852897644043 time for calcul the mask position with numpy : 0.024595975875854492 nb_pixel_total : 87277 time to create 1 rle with old method : 0.09221410751342773 create new chi : 1.538344144821167 time to delete rle : 0.0006730556488037109 batch 1 Loaded 9 chid ids of type : 3594 ++++++++Number RLEs to save : 4630 TO DO : save crop sub photo not yet done ! save time : 0.31508612632751465 nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 5.74700665473938 time for calcul the mask position with numpy : 0.48140740394592285 nb_pixel_total : 7556591 time to create 1 rle with new method : 0.8428542613983154 time for calcul the mask position with numpy : 0.023290395736694336 nb_pixel_total : 23084 time to create 1 rle with old method : 0.024137258529663086 time for calcul the mask position with numpy : 0.02344822883605957 nb_pixel_total : 56570 time to create 1 rle with old method : 0.058371543884277344 time for calcul the mask position with numpy : 0.024511337280273438 nb_pixel_total : 34331 time to create 1 rle with old method : 0.03669476509094238 time for calcul the mask position with numpy : 0.023103952407836914 nb_pixel_total : 18238 time to create 1 rle with old method : 0.018949270248413086 time for calcul the mask position with numpy : 0.023560523986816406 nb_pixel_total : 28716 time to create 1 rle with old method : 0.029958486557006836 time for calcul the mask position with numpy : 0.02449631690979004 nb_pixel_total : 54171 time to create 1 rle with old method : 0.0565488338470459 time for calcul the mask position with numpy : 0.023767471313476562 nb_pixel_total : 37573 time to create 1 rle with old method : 0.03900146484375 time for calcul the mask position with numpy : 0.023462533950805664 nb_pixel_total : 39924 time to create 1 rle with old method : 0.04087638854980469 time for calcul the mask position with numpy : 0.024383544921875 nb_pixel_total : 201058 time to create 1 rle with new method : 0.5987191200256348 time for calcul the mask position with numpy : 0.02445054054260254 nb_pixel_total : 34405 time to create 1 rle with old method : 0.035804033279418945 time for calcul the mask position with numpy : 0.023181915283203125 nb_pixel_total : 23410 time to create 1 rle with old method : 0.024546384811401367 time for calcul the mask position with numpy : 0.02487468719482422 nb_pixel_total : 58749 time to create 1 rle with old method : 0.06135368347167969 time for calcul the mask position with numpy : 0.024086713790893555 nb_pixel_total : 26947 time to create 1 rle with old method : 0.027455806732177734 time for calcul the mask position with numpy : 0.02418804168701172 nb_pixel_total : 53165 time to create 1 rle with old method : 0.055898189544677734 time for calcul the mask position with numpy : 0.02360677719116211 nb_pixel_total : 38021 time to create 1 rle with old method : 0.03925061225891113 time for calcul the mask position with numpy : 0.023736238479614258 nb_pixel_total : 9447 time to create 1 rle with old method : 0.00981760025024414 create new chi : 2.9306724071502686 time to delete rle : 0.001466512680053711 batch 1 Loaded 33 chid ids of type : 3594 +++++++++++++++++++++++++Number RLEs to save : 11651 TO DO : save crop sub photo not yet done ! save time : 0.7358353137969971 map_output_result : {1381228491: (0.0, 'Should be the crop_list due to order', 0), 1381228489: (0.0, 'Should be the crop_list due to order', 0), 1381228477: (0.0, 'Should be the crop_list due to order', 0), 1381228426: (0.0, 'Should be the crop_list due to order', 0), 1381228415: (0.0, 'Should be the crop_list due to order', 0), 1381228412: (0.0, 'Should be the crop_list due to order', 0), 1381228403: (0.0, 'Should be the crop_list due to order', 0), 1381228399: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1381228491, 1381228489, 1381228477, 1381228426, 1381228415, 1381228412, 1381228403, 1381228399] Looping around the photos to save general results len do output : 8 /1381228491.Didn't retrieve data . /1381228489.Didn't retrieve data . /1381228477.Didn't retrieve data . /1381228426.Didn't retrieve data . /1381228415.Didn't retrieve data . /1381228412.Didn't retrieve data . /1381228403.Didn't retrieve data . /1381228399.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228491', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228489', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228477', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228426', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228415', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228412', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228403', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228399', None, None, None, None, None, '3660766') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.012781143188476562 save_final save missing photos in datou_result : time spend for datou_step_exec : 50.683815717697144 time spend to save output : 0.013156652450561523 total time spend for step 3 : 50.696972370147705 step4:ventilate_hashtags_in_portfolio Wed Sep 3 21:42:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26504438 get user id for portfolio 26504438 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26504438 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','background','autre','flou','metal','papier','pehd','environnement','mal_croppe','pet_fonce','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26504438 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','background','autre','flou','metal','papier','pehd','environnement','mal_croppe','pet_fonce','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26504438 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_clair','background','autre','flou','metal','papier','pehd','environnement','mal_croppe','pet_fonce','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26504733,26504734,26504735,26504736,26504737,26504738,26504739,26504740,26504741,26504742,26504743?tags=pet_clair,background,autre,flou,metal,papier,pehd,environnement,mal_croppe,pet_fonce,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1381228491, 1381228489, 1381228477, 1381228426, 1381228415, 1381228412, 1381228403, 1381228399] Looping around the photos to save general results len do output : 1 /26504438. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228491', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228489', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228477', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228426', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228415', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228412', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228403', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228399', None, None, None, None, None, '3660766') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.019911527633666992 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6542017459869385 time spend to save output : 0.020185232162475586 total time spend for step 4 : 1.674386978149414 step5:final Wed Sep 3 21:42:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1381228491: ('0.04470118393132717',), 1381228489: ('0.04470118393132717',), 1381228477: ('0.04470118393132717',), 1381228426: ('0.04470118393132717',), 1381228415: ('0.04470118393132717',), 1381228412: ('0.04470118393132717',), 1381228403: ('0.04470118393132717',), 1381228399: ('0.04470118393132717',)} new output for save of step final : {1381228491: ('0.04470118393132717',), 1381228489: ('0.04470118393132717',), 1381228477: ('0.04470118393132717',), 1381228426: ('0.04470118393132717',), 1381228415: ('0.04470118393132717',), 1381228412: ('0.04470118393132717',), 1381228403: ('0.04470118393132717',), 1381228399: ('0.04470118393132717',)} [1381228491, 1381228489, 1381228477, 1381228426, 1381228415, 1381228412, 1381228403, 1381228399] Looping around the photos to save general results len do output : 8 /1381228491.Didn't retrieve data . /1381228489.Didn't retrieve data . /1381228477.Didn't retrieve data . /1381228426.Didn't retrieve data . /1381228415.Didn't retrieve data . /1381228412.Didn't retrieve data . /1381228403.Didn't retrieve data . /1381228399.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228491', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228489', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228477', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228426', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228415', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228412', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228403', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228399', None, None, None, None, None, '3660766') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.013367176055908203 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12335562705993652 time spend to save output : 0.01381230354309082 total time spend for step 5 : 0.13716793060302734 step6:blur_detection Wed Sep 3 21:42:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c.jpg resize: (2160, 3840) 1381228491 -6.968304728409827 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13.jpg resize: (2160, 3840) 1381228489 -6.9766998277749925 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495.jpg resize: (2160, 3840) 1381228477 -6.975636924144841 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f.jpg resize: (2160, 3840) 1381228426 -7.140519617429194 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2.jpg resize: (2160, 3840) 1381228415 -6.922720263116028 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c.jpg resize: (2160, 3840) 1381228412 -6.890467031154438 treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea.jpg resize: (2160, 3840) 1381228403 -7.0881517207134275 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca.jpg resize: (2160, 3840) 1381228399 -6.900233901449682 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987346_0.png resize: (157, 137) 1381249070 -4.373187489292725 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987347_0.png resize: (225, 232) 1381249071 -3.0690282245119644 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987348_0.png resize: (231, 337) 1381249072 -4.582302662859858 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987350_0.png resize: (167, 200) 1381249074 -4.166692498602725 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987351_0.png resize: (359, 541) 1381249075 -4.150433830822757 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987352_0.png resize: (163, 178) 1381249076 -3.7803595878181717 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987353_0.png resize: (320, 177) 1381249077 -4.257061739227541 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987354_0.png resize: (353, 405) 1381249079 -4.6179914103668125 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987355_0.png resize: (294, 303) 1381249080 -2.6353151969997084 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987356_0.png resize: (204, 126) 1381249081 -3.077088426740274 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987358_0.png resize: (261, 312) 1381249083 -3.023823604379765 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987360_0.png resize: (299, 375) 1381249084 -4.991830547985821 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987361_0.png resize: (169, 203) 1381249085 -3.8270388445089925 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987362_0.png resize: (113, 133) 1381249087 -3.524010556878463 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987364_0.png resize: (136, 73) 1381249088 -3.506091954376815 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987366_0.png resize: (191, 251) 1381249089 0.06213920915863641 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987369_0.png resize: (269, 163) 1381249091 -4.633391323006989 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987370_0.png resize: (269, 190) 1381249092 -5.191615749655075 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987371_0.png resize: (112, 129) 1381249093 -1.9870543572773023 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987372_0.png resize: (215, 156) 1381249095 -3.825945821348801 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987374_0.png resize: (181, 237) 1381249096 -2.5968408468225097 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987375_0.png resize: (101, 129) 1381249097 -3.0194229368798453 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987376_0.png resize: (144, 236) 1381249098 -3.223341200406234 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987377_0.png resize: (284, 160) 1381249099 -5.013625842250197 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987378_0.png resize: (232, 160) 1381249100 -4.114436755800154 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987379_0.png resize: (99, 94) 1381249101 -1.9417356815686206 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987381_0.png resize: (398, 371) 1381249102 -5.83110514563886 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987382_0.png resize: (178, 218) 1381249103 -0.39538772610470074 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987384_0.png resize: (314, 92) 1381249104 -3.003530101358686 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987385_0.png resize: (192, 227) 1381249105 -3.112825950057601 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987386_0.png resize: (260, 134) 1381249106 -3.4063966737585156 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987388_0.png resize: (150, 271) 1381249107 -3.129887711999446 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987389_0.png resize: (184, 112) 1381249108 -2.966102023514447 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987390_0.png resize: (127, 118) 1381249109 -1.61619424463719 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987391_0.png resize: (182, 289) 1381249110 -5.438520142901336 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987392_0.png resize: (315, 244) 1381249111 -3.9568744277912717 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987395_0.png resize: (274, 283) 1381249112 -3.7747890223944554 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987396_0.png resize: (299, 330) 1381249113 -4.376445113955079 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987397_0.png resize: (307, 200) 1381249114 -2.188918551402704 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987398_0.png resize: (219, 189) 1381249115 -4.051821373126325 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987399_0.png resize: (89, 179) 1381249116 -3.716397882442261 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987400_0.png resize: (142, 176) 1381249117 -2.566049990164915 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987401_0.png resize: (420, 376) 1381249118 -4.733514508155825 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987403_0.png resize: (244, 190) 1381249119 -4.277275571243195 treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987404_0.png resize: (283, 554) 1381249120 -4.2479451648372075 treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987405_0.png resize: (262, 418) 1381249121 -3.042951227747125 treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987406_0.png resize: (192, 226) 1381249122 -3.32233698310696 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987409_0.png resize: (254, 237) 1381249123 -4.1856242609608785 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987410_0.png resize: (298, 271) 1381249124 -2.3351501982473506 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987411_0.png resize: (234, 209) 1381249125 -3.5468795267090476 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987414_0.png resize: (290, 174) 1381249126 -4.128127275669116 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987415_0.png resize: (550, 707) 1381249127 -4.536197245071917 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987416_0.png resize: (315, 198) 1381249128 -4.518276364313253 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987417_0.png resize: (189, 299) 1381249129 -4.1189815778196675 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987418_0.png resize: (271, 341) 1381249130 -2.8242141076625806 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987419_0.png resize: (229, 273) 1381249131 -4.097569080745191 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987420_0.png resize: (178, 197) 1381249132 -4.5920902566251325 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987421_0.png resize: (299, 212) 1381249133 -4.438294321013845 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987422_0.png resize: (328, 305) 1381249134 -3.646409625419533 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987423_0.png resize: (187, 171) 1381249135 -4.161513737472427 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987367_0.png resize: (109, 171) 1381249141 -2.642317881981992 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987368_0.png resize: (175, 339) 1381249142 -3.356942056514609 treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987373_0.png resize: (604, 248) 1381249143 -3.1440624869926284 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987380_0.png resize: (547, 233) 1381249144 -3.601583718699099 treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987383_0.png resize: (143, 175) 1381249145 -2.6962740426990712 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987394_0.png resize: (222, 174) 1381249146 -3.5551666858526603 treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987407_0.png resize: (225, 261) 1381249148 -2.9365379230322612 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987408_0.png resize: (98, 154) 1381249149 -2.1358614926951422 treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987349_0.png resize: (349, 308) 1381249152 -4.305782687559662 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987357_0.png resize: (181, 212) 1381249153 -3.9683416891826226 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987359_0.png resize: (63, 199) 1381249154 -3.115143826867871 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987363_0.png resize: (135, 171) 1381249155 -3.3030594506367734 treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987365_0.png resize: (349, 232) 1381249156 -3.576675206518881 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987387_0.png resize: (82, 129) 1381249157 -2.007618896639321 treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987393_0.png resize: (242, 150) 1381249158 -4.505519543143171 treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987402_0.png resize: (111, 444) 1381249159 -4.225379718168313 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987412_0.png resize: (355, 265) 1381249160 -4.850723812146929 treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987413_0.png resize: (176, 211) 1381249161 -2.276634679186068 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 86 time used for this insertion : 0.016545534133911133 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 86 time used for this insertion : 0.017144203186035156 save missing photos in datou_result : time spend for datou_step_exec : 29.424232244491577 time spend to save output : 0.03839516639709473 total time spend for step 6 : 29.462627410888672 step7:brightness Wed Sep 3 21:43:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c.jpg treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13.jpg treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495.jpg treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f.jpg treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2.jpg treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c.jpg treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea.jpg treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca.jpg treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987346_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987347_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987348_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987350_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987351_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987352_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987353_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987354_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987355_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987356_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987358_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987360_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987361_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987362_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987364_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987366_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987369_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987370_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987371_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987372_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987374_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987375_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987376_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987377_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987378_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987379_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987381_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987382_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987384_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987385_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987386_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987388_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987389_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987390_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987391_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987392_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987395_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987396_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987397_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987398_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987399_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987400_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987401_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987403_0.png treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987404_0.png treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987405_0.png treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987406_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987409_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987410_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987411_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987414_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987415_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987416_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987417_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987418_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987419_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987420_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987421_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987422_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987423_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987367_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987368_0.png treat image : temp/1756928429_3172014_1381228477_2986a4b722376afa628f55e548a85495_rle_crop_3942987373_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987380_0.png treat image : temp/1756928429_3172014_1381228426_9add86ac9ccb27c50a2f2cc9ff12ac8f_rle_crop_3942987383_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987394_0.png treat image : temp/1756928429_3172014_1381228403_331db3a906f94d93c59acb5b5b1809ea_rle_crop_3942987407_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987408_0.png treat image : temp/1756928429_3172014_1381228491_407dad81ff01a87e38f562b30502745c_rle_crop_3942987349_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987357_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987359_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987363_0.png treat image : temp/1756928429_3172014_1381228489_987155c33d0af4ab4f15ae544427ac13_rle_crop_3942987365_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987387_0.png treat image : temp/1756928429_3172014_1381228415_39990657aa26534927cec543206171e2_rle_crop_3942987393_0.png treat image : temp/1756928429_3172014_1381228412_3f42d4736e2c1e2cb41484f8772f290c_rle_crop_3942987402_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987412_0.png treat image : temp/1756928429_3172014_1381228399_e3c06be9d0e0beb38b3d7586d64526ca_rle_crop_3942987413_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 86 time used for this insertion : 0.017012834548950195 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 86 time used for this insertion : 0.017076969146728516 save missing photos in datou_result : time spend for datou_step_exec : 7.631412029266357 time spend to save output : 0.0384364128112793 total time spend for step 7 : 7.669848442077637 step8:velours_tree Wed Sep 3 21:43:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.10980725288391113 time spend to save output : 4.38690185546875e-05 total time spend for step 8 : 0.10985112190246582 step9:send_mail_cod Wed Sep 3 21:43:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26504438_03-09-2025_21_43_35.pdf 26504733 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265047331756928615 26504734 imagette265047341756928616 26504735 change filename to text .imagette265047351756928616 26504736 imagette265047361756928616 26504737 imagette265047371756928616 26504738 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265047381756928616 26504739 imagette265047391756928617 26504741 imagette265047411756928617 26504742 imagette265047421756928617 26504743 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265047431756928617 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26504438 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26504733,26504734,26504735,26504736,26504737,26504738,26504739,26504740,26504741,26504742,26504743?tags=pet_clair,background,autre,flou,metal,papier,pehd,environnement,mal_croppe,pet_fonce,carton args[1381228491] : ((1381228491, -6.968304728409827, 492609224), (1381228491, 0.4579158862658159, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228489] : ((1381228489, -6.9766998277749925, 492609224), (1381228489, 0.29604100441651393, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228477] : ((1381228477, -6.975636924144841, 492609224), (1381228477, 0.26729234731444024, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228426] : ((1381228426, -7.140519617429194, 492609224), (1381228426, 0.31229411751122144, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228415] : ((1381228415, -6.922720263116028, 492609224), (1381228415, 0.33185282298053, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228412] : ((1381228412, -6.890467031154438, 492609224), (1381228412, 0.650629658694278, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228403] : ((1381228403, -7.0881517207134275, 492609224), (1381228403, 0.9426102623870312, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com args[1381228399] : ((1381228399, -6.900233901449682, 492609224), (1381228399, 0.5118607661870667, 2107752395), '0.04470118393132717') We are sending mail with results at report@fotonower.com refus_total : 0.04470118393132717 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26504438 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26504438_03-09-2025_21_43_35.pdf results_Auto_P26504438_03-09-2025_21_43_35.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26504438_03-09-2025_21_43_35.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26504438','results_Auto_P26504438_03-09-2025_21_43_35.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26504438_03-09-2025_21_43_35.pdf','pdf','','0.45','0.04470118393132717') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26504438

https://www.fotonower.com/image?json=false&list_photos_id=1381228491
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228489
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228477
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228426
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228415
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228412
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228403
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1381228399
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 4.47%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/26504733?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/26504735?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26504738?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26504743?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26504438_03-09-2025_21_43_35.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/26504733,26504734,26504735,26504736,26504737,26504738,26504739,26504740,26504741,26504742,26504743?tags=pet_clair,background,autre,flou,metal,papier,pehd,environnement,mal_croppe,pet_fonce,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 03 Sep 2025 19:43:45 GMT Content-Length: 0 Connection: close X-Message-Id: k7zP4FONT6qQhyX47C2tgA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1381228491, 1381228489, 1381228477, 1381228426, 1381228415, 1381228412, 1381228403, 1381228399] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228491', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228489', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228477', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228426', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228415', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228412', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228403', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228399', None, None, None, None, None, '3660766') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.024092435836791992 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.600664854049683 time spend to save output : 0.024292945861816406 total time spend for step 9 : 10.624957799911499 step10:split_time_score Wed Sep 3 21:43:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 03092025 26504438 Nombre de photos uploadées : 8 / 23040 (0%) 03092025 26504438 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 03092025 26504438 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0004291534423828125 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.2550199031829834 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.033641923365876306 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26474311_03-09-2025_07_36_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26474311 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26474311 AND mptpi.`type`=3726 To do Qualite : 0.05173950027863514 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26492268_03-09-2025_15_42_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26492268 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26492268 AND mptpi.`type`=3594 To do Qualite : 0.03993974264039564 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26479246_03-09-2025_10_13_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26479246 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26479246 AND mptpi.`type`=3726 To do Qualite : 0.040500265772951315 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26482157_03-09-2025_11_12_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26482157 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26482157 AND mptpi.`type`=3726 To do Qualite : 0.021976532867700977 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26482163_03-09-2025_11_06_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26482163 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26482163 AND mptpi.`type`=3726 To do Qualite : 0.044307632490128215 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26491437_03-09-2025_20_47_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26491437 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26491437 AND mptpi.`type`=3726 To do Qualite : 0.06122043788580245 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26487565_03-09-2025_13_42_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26487565 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26487565 AND mptpi.`type`=3594 To do Qualite : 0.06982871977880656 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26489096_03-09-2025_14_17_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26489096 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26489096 AND mptpi.`type`=3594 To do Qualite : 0.028099811787241708 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26489121_03-09-2025_14_16_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26489121 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26489121 AND mptpi.`type`=3726 To do Qualite : 0.11959475396324351 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26492294_03-09-2025_15_46_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26492294 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26492294 AND mptpi.`type`=3594 To do Qualite : 0.015028770592978696 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26495360_03-09-2025_17_13_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26495360 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26495360 AND mptpi.`type`=3726 To do Qualite : 0.07952586484053498 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26495364_03-09-2025_17_16_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26495364 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26495364 AND mptpi.`type`=3594 To do Qualite : 0.047140260473402334 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26497815_03-09-2025_18_47_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26497815 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26497815 AND mptpi.`type`=3594 To do Qualite : 0.04470118393132717 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26504438_03-09-2025_21_43_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26504438 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26504438 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'03092025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1381228491, 1381228489, 1381228477, 1381228426, 1381228415, 1381228412, 1381228403, 1381228399] Looping around the photos to save general results len do output : 1 /26504438Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228491', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228489', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228477', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228426', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228415', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228412', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228403', None, None, None, None, None, '3660766') ('3318', None, None, None, None, None, None, None, '3660766') ('3318', '26504438', '1381228399', None, None, None, None, None, '3660766') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013799905776977539 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.3275198936462402 time spend to save output : 0.013975381851196289 total time spend for step 10 : 1.3414952754974365 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 86.11user 74.01system 3:22.12elapsed 79%CPU (0avgtext+0avgdata 3954344maxresident)k 618088inputs+70848outputs (3602major+8309758minor)pagefaults 0swaps