python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3332805 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3729397'] with mtr_portfolio_ids : ['26902191'] and first list_photo_ids : [] new path : /proc/3332805/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 12 ; length of list_pids : 12 ; length of list_args : 12 time to download the photos : 1.8728742599487305 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Sep 16 18:00:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10589 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-16 18:00:31.551579: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-16 18:00:31.576575: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-16 18:00:31.578669: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f258c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-16 18:00:31.578717: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-16 18:00:31.582405: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-16 18:00:31.757038: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x715b8b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-16 18:00:31.757081: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-16 18:00:31.758445: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-16 18:00:31.758833: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-16 18:00:31.761675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-16 18:00:31.764549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-16 18:00:31.765029: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-16 18:00:31.767964: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-16 18:00:31.768979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-16 18:00:31.773620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-16 18:00:31.775038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-16 18:00:31.775101: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-16 18:00:31.775804: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-16 18:00:31.775818: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-16 18:00:31.775825: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-16 18:00:31.777190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9808 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-16 18:00:32.043534: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-16 18:00:32.043625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-16 18:00:32.043653: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-16 18:00:32.043677: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-16 18:00:32.043701: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-16 18:00:32.043725: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-16 18:00:32.043749: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-16 18:00:32.043773: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-16 18:00:32.045908: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-16 18:00:32.047169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-16 18:00:32.047201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-16 18:00:32.047216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-16 18:00:32.047230: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-16 18:00:32.047244: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-16 18:00:32.047258: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-16 18:00:32.047270: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-16 18:00:32.047283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-16 18:00:32.048577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-16 18:00:32.048605: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-16 18:00:32.048614: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-16 18:00:32.048622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-16 18:00:32.049899: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9808 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-16 18:00:39.780805: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-16 18:00:39.966285: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 41.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 45.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 40.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 41.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 36.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 13 Detection mask done ! Trying to reset tf kernel 3333687 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4498 tf kernel not reseted sub process len(results) : 12 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 12 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9787 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00021982192993164062 nb_pixel_total : 2927 time to create 1 rle with old method : 0.0034856796264648438 length of segment : 45 time for calcul the mask position with numpy : 0.00037288665771484375 nb_pixel_total : 10325 time to create 1 rle with old method : 0.011931657791137695 length of segment : 182 time for calcul the mask position with numpy : 0.001491546630859375 nb_pixel_total : 46783 time to create 1 rle with old method : 0.051308631896972656 length of segment : 298 time for calcul the mask position with numpy : 0.0004928112030029297 nb_pixel_total : 15618 time to create 1 rle with old method : 0.017708778381347656 length of segment : 170 time for calcul the mask position with numpy : 0.00034332275390625 nb_pixel_total : 9529 time to create 1 rle with old method : 0.01064157485961914 length of segment : 131 time for calcul the mask position with numpy : 0.0005817413330078125 nb_pixel_total : 13657 time to create 1 rle with old method : 0.015191793441772461 length of segment : 184 time for calcul the mask position with numpy : 0.00027108192443847656 nb_pixel_total : 16547 time to create 1 rle with old method : 0.018636465072631836 length of segment : 177 time for calcul the mask position with numpy : 0.0018770694732666016 nb_pixel_total : 105759 time to create 1 rle with old method : 0.12031912803649902 length of segment : 526 time for calcul the mask position with numpy : 0.00013184547424316406 nb_pixel_total : 5811 time to create 1 rle with old method : 0.006687164306640625 length of segment : 81 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 11043 time to create 1 rle with old method : 0.012852191925048828 length of segment : 151 time for calcul the mask position with numpy : 0.00043392181396484375 nb_pixel_total : 22932 time to create 1 rle with old method : 0.026002168655395508 length of segment : 197 time for calcul the mask position with numpy : 0.0001671314239501953 nb_pixel_total : 4236 time to create 1 rle with old method : 0.004863739013671875 length of segment : 77 time for calcul the mask position with numpy : 0.00011849403381347656 nb_pixel_total : 7088 time to create 1 rle with old method : 0.00840449333190918 length of segment : 58 time for calcul the mask position with numpy : 0.00035691261291503906 nb_pixel_total : 5724 time to create 1 rle with old method : 0.006789684295654297 length of segment : 118 time for calcul the mask position with numpy : 0.00029969215393066406 nb_pixel_total : 11749 time to create 1 rle with old method : 0.0135498046875 length of segment : 159 time for calcul the mask position with numpy : 0.00013637542724609375 nb_pixel_total : 4715 time to create 1 rle with old method : 0.0057790279388427734 length of segment : 39 time for calcul the mask position with numpy : 0.00013375282287597656 nb_pixel_total : 3623 time to create 1 rle with old method : 0.004446268081665039 length of segment : 68 time for calcul the mask position with numpy : 0.00015735626220703125 nb_pixel_total : 4280 time to create 1 rle with old method : 0.005105733871459961 length of segment : 60 time for calcul the mask position with numpy : 0.00015044212341308594 nb_pixel_total : 6511 time to create 1 rle with old method : 0.007448911666870117 length of segment : 69 time for calcul the mask position with numpy : 0.002630472183227539 nb_pixel_total : 89842 time to create 1 rle with old method : 0.10015225410461426 length of segment : 558 time for calcul the mask position with numpy : 0.00037384033203125 nb_pixel_total : 9202 time to create 1 rle with old method : 0.01036381721496582 length of segment : 107 time for calcul the mask position with numpy : 0.002483844757080078 nb_pixel_total : 55353 time to create 1 rle with old method : 0.06021475791931152 length of segment : 352 time for calcul the mask position with numpy : 0.00027370452880859375 nb_pixel_total : 5486 time to create 1 rle with old method : 0.007672309875488281 length of segment : 70 time for calcul the mask position with numpy : 0.00016355514526367188 nb_pixel_total : 6183 time to create 1 rle with old method : 0.00706791877746582 length of segment : 115 time for calcul the mask position with numpy : 0.034386396408081055 nb_pixel_total : 699774 time to create 1 rle with new method : 0.04488110542297363 length of segment : 1096 time for calcul the mask position with numpy : 0.00011181831359863281 nb_pixel_total : 2941 time to create 1 rle with old method : 0.005101680755615234 length of segment : 44 time for calcul the mask position with numpy : 0.00045299530029296875 nb_pixel_total : 13091 time to create 1 rle with old method : 0.021727800369262695 length of segment : 164 time for calcul the mask position with numpy : 0.00017023086547851562 nb_pixel_total : 3300 time to create 1 rle with old method : 0.004816770553588867 length of segment : 44 time for calcul the mask position with numpy : 0.016429662704467773 nb_pixel_total : 1217054 time to create 1 rle with new method : 0.04789280891418457 length of segment : 1086 time for calcul the mask position with numpy : 0.00021266937255859375 nb_pixel_total : 12346 time to create 1 rle with old method : 0.014423608779907227 length of segment : 121 time for calcul the mask position with numpy : 0.00030612945556640625 nb_pixel_total : 19265 time to create 1 rle with old method : 0.02202773094177246 length of segment : 143 time for calcul the mask position with numpy : 0.00012755393981933594 nb_pixel_total : 4273 time to create 1 rle with old method : 0.005069255828857422 length of segment : 67 time for calcul the mask position with numpy : 0.0002608299255371094 nb_pixel_total : 7338 time to create 1 rle with old method : 0.01015019416809082 length of segment : 92 time for calcul the mask position with numpy : 0.00031113624572753906 nb_pixel_total : 9237 time to create 1 rle with old method : 0.010822057723999023 length of segment : 103 time for calcul the mask position with numpy : 0.0003719329833984375 nb_pixel_total : 14892 time to create 1 rle with old method : 0.01695704460144043 length of segment : 122 time for calcul the mask position with numpy : 0.0004711151123046875 nb_pixel_total : 9750 time to create 1 rle with old method : 0.01093292236328125 length of segment : 177 time for calcul the mask position with numpy : 0.027007579803466797 nb_pixel_total : 1016387 time to create 1 rle with new method : 0.05868268013000488 length of segment : 1432 time for calcul the mask position with numpy : 0.0002627372741699219 nb_pixel_total : 9715 time to create 1 rle with old method : 0.011262655258178711 length of segment : 123 time for calcul the mask position with numpy : 0.00011849403381347656 nb_pixel_total : 3756 time to create 1 rle with old method : 0.004582405090332031 length of segment : 67 time for calcul the mask position with numpy : 0.00043702125549316406 nb_pixel_total : 13013 time to create 1 rle with old method : 0.015239953994750977 length of segment : 119 time for calcul the mask position with numpy : 0.00022411346435546875 nb_pixel_total : 11381 time to create 1 rle with old method : 0.013319253921508789 length of segment : 107 time for calcul the mask position with numpy : 0.00011491775512695312 nb_pixel_total : 3580 time to create 1 rle with old method : 0.004358530044555664 length of segment : 67 time for calcul the mask position with numpy : 0.00019860267639160156 nb_pixel_total : 4174 time to create 1 rle with old method : 0.004866361618041992 length of segment : 103 time for calcul the mask position with numpy : 9.202957153320312e-05 nb_pixel_total : 2896 time to create 1 rle with old method : 0.0033669471740722656 length of segment : 66 time for calcul the mask position with numpy : 8.106231689453125e-05 nb_pixel_total : 2293 time to create 1 rle with old method : 0.0027472972869873047 length of segment : 57 time for calcul the mask position with numpy : 9.036064147949219e-05 nb_pixel_total : 3240 time to create 1 rle with old method : 0.003722667694091797 length of segment : 58 time for calcul the mask position with numpy : 0.00018024444580078125 nb_pixel_total : 8993 time to create 1 rle with old method : 0.010039091110229492 length of segment : 112 time for calcul the mask position with numpy : 9.846687316894531e-05 nb_pixel_total : 2047 time to create 1 rle with old method : 0.0024645328521728516 length of segment : 75 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 9037 time to create 1 rle with old method : 0.01043558120727539 length of segment : 127 time for calcul the mask position with numpy : 8.702278137207031e-05 nb_pixel_total : 2537 time to create 1 rle with old method : 0.0029897689819335938 length of segment : 56 time spent for convertir_results : 3.373250722885132 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 50 chid ids of type : 3594 Number RLEs to save : 9820 save missing photos in datou_result : time spend for datou_step_exec : 26.26430034637451 time spend to save output : 0.6649289131164551 total time spend for step 1 : 26.929229259490967 step2:crop_condition Tue Sep 16 18:00:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 12 ! batch 1 Loaded 50 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 39 About to insert : list_path_to_insert length 39 new photo from crops ! About to upload 39 photos upload in portfolio : 3736932 init cache_photo without model_param we have 39 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758038457_3332805 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 39 photos in the portfolio 3736932 time of upload the photos Elapsed time : 9.959340572357178 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758038468_3332805 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5487816333770752 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758038469_3332805 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5474634170532227 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3736932 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758038483_3332805 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 3736932 time of upload the photos Elapsed time : 6.484717130661011 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1383950821, 1383950819, 1383950816, 1383950811, 1383950589, 1383950588, 1383950585, 1383950582, 1383950578, 1383950576, 1383950552, 1383950547] Looping around the photos to save general results len do output : 50 /1383955402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383955879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950821', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950819', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950816', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950811', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950589', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950588', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950585', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950582', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950578', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950576', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950552', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950547', None, None, None, None, None, '3729397') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 162 time used for this insertion : 0.022960424423217773 save_final save missing photos in datou_result : time spend for datou_step_exec : 33.508169651031494 time spend to save output : 0.025359392166137695 total time spend for step 2 : 33.53352904319763 step3:rle_unique_nms_with_priority Tue Sep 16 18:01:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 50 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.4968693256378174 time for calcul the mask position with numpy : 0.16885137557983398 nb_pixel_total : 1974761 time to create 1 rle with new method : 0.07988691329956055 time for calcul the mask position with numpy : 0.006136178970336914 nb_pixel_total : 13657 time to create 1 rle with old method : 0.015271425247192383 time for calcul the mask position with numpy : 0.006171703338623047 nb_pixel_total : 9529 time to create 1 rle with old method : 0.010479450225830078 time for calcul the mask position with numpy : 0.0062408447265625 nb_pixel_total : 15618 time to create 1 rle with old method : 0.016654014587402344 time for calcul the mask position with numpy : 0.006573677062988281 nb_pixel_total : 46783 time to create 1 rle with old method : 0.04969668388366699 time for calcul the mask position with numpy : 0.0059583187103271484 nb_pixel_total : 10325 time to create 1 rle with old method : 0.011028766632080078 time for calcul the mask position with numpy : 0.005931854248046875 nb_pixel_total : 2927 time to create 1 rle with old method : 0.003137826919555664 create new chi : 0.40251946449279785 time to delete rle : 0.026812076568603516 batch 1 Loaded 13 chid ids of type : 3594 ++++++++++++Number RLEs to save : 3100 TO DO : save crop sub photo not yet done ! save time : 0.24698829650878906 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.2710397243499756 time for calcul the mask position with numpy : 0.054083824157714844 nb_pixel_total : 1934440 time to create 1 rle with new method : 0.19714593887329102 time for calcul the mask position with numpy : 0.00650334358215332 nb_pixel_total : 11043 time to create 1 rle with old method : 0.012312889099121094 time for calcul the mask position with numpy : 0.0060961246490478516 nb_pixel_total : 5811 time to create 1 rle with old method : 0.006524324417114258 time for calcul the mask position with numpy : 0.006638765335083008 nb_pixel_total : 105759 time to create 1 rle with old method : 0.11724615097045898 time for calcul the mask position with numpy : 0.006569862365722656 nb_pixel_total : 16547 time to create 1 rle with old method : 0.018474340438842773 create new chi : 0.4321479797363281 time to delete rle : 0.0005249977111816406 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 2950 TO DO : save crop sub photo not yet done ! save time : 0.23115229606628418 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.06062054634094238 time for calcul the mask position with numpy : 0.023531198501586914 nb_pixel_total : 2039344 time to create 1 rle with new method : 0.14756202697753906 time for calcul the mask position with numpy : 0.006108999252319336 nb_pixel_total : 7088 time to create 1 rle with old method : 0.007964372634887695 time for calcul the mask position with numpy : 0.006081342697143555 nb_pixel_total : 4236 time to create 1 rle with old method : 0.004637956619262695 time for calcul the mask position with numpy : 0.006265878677368164 nb_pixel_total : 22932 time to create 1 rle with old method : 0.029902219772338867 create new chi : 0.23243093490600586 time to delete rle : 0.00033164024353027344 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1744 TO DO : save crop sub photo not yet done ! save time : 0.16191887855529785 nb_obj : 13 nb_hashtags : 2 time to prepare the origin masks : 0.573110818862915 time for calcul the mask position with numpy : 0.03980135917663574 nb_pixel_total : 1179335 time to create 1 rle with new method : 0.16959810256958008 time for calcul the mask position with numpy : 0.006361961364746094 nb_pixel_total : 2941 time to create 1 rle with old method : 0.0033702850341796875 time for calcul the mask position with numpy : 0.014456748962402344 nb_pixel_total : 699774 time to create 1 rle with new method : 0.1669454574584961 time for calcul the mask position with numpy : 0.0065288543701171875 nb_pixel_total : 6183 time to create 1 rle with old method : 0.007604122161865234 time for calcul the mask position with numpy : 0.006641864776611328 nb_pixel_total : 5486 time to create 1 rle with old method : 0.006587982177734375 time for calcul the mask position with numpy : 0.0067386627197265625 nb_pixel_total : 44235 time to create 1 rle with old method : 0.05277585983276367 time for calcul the mask position with numpy : 0.006595611572265625 nb_pixel_total : 9202 time to create 1 rle with old method : 0.011061429977416992 time for calcul the mask position with numpy : 0.00685572624206543 nb_pixel_total : 89842 time to create 1 rle with old method : 0.10433340072631836 time for calcul the mask position with numpy : 0.006384611129760742 nb_pixel_total : 6511 time to create 1 rle with old method : 0.00748443603515625 time for calcul the mask position with numpy : 0.006562471389770508 nb_pixel_total : 4280 time to create 1 rle with old method : 0.004825115203857422 time for calcul the mask position with numpy : 0.006431102752685547 nb_pixel_total : 3623 time to create 1 rle with old method : 0.004359245300292969 time for calcul the mask position with numpy : 0.006591081619262695 nb_pixel_total : 4715 time to create 1 rle with old method : 0.005495309829711914 time for calcul the mask position with numpy : 0.006500720977783203 nb_pixel_total : 11749 time to create 1 rle with old method : 0.01354837417602539 time for calcul the mask position with numpy : 0.006991386413574219 nb_pixel_total : 5724 time to create 1 rle with old method : 0.006430149078369141 create new chi : 0.708362340927124 time to delete rle : 0.0008120536804199219 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 6726 TO DO : save crop sub photo not yet done ! save time : 0.4621894359588623 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.06136679649353027 time for calcul the mask position with numpy : 0.013695955276489258 nb_pixel_total : 856546 time to create 1 rle with new method : 0.03382682800292969 time for calcul the mask position with numpy : 0.06334805488586426 nb_pixel_total : 1200663 time to create 1 rle with new method : 0.10169577598571777 time for calcul the mask position with numpy : 0.007570028305053711 nb_pixel_total : 3300 time to create 1 rle with old method : 0.0037190914154052734 time for calcul the mask position with numpy : 0.006127834320068359 nb_pixel_total : 13091 time to create 1 rle with old method : 0.014804363250732422 create new chi : 0.28190112113952637 time to delete rle : 0.0005435943603515625 batch 1 Loaded 8 chid ids of type : 3594 ++++Number RLEs to save : 3668 TO DO : save crop sub photo not yet done ! save time : 0.25174975395202637 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.052286624908447266 time for calcul the mask position with numpy : 0.02293562889099121 nb_pixel_total : 2050062 time to create 1 rle with new method : 0.08453202247619629 time for calcul the mask position with numpy : 0.006410837173461914 nb_pixel_total : 4273 time to create 1 rle with old method : 0.004866838455200195 time for calcul the mask position with numpy : 0.006406307220458984 nb_pixel_total : 19265 time to create 1 rle with old method : 0.02174210548400879 create new chi : 0.15464043617248535 time to delete rle : 0.00023937225341796875 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1500 TO DO : save crop sub photo not yet done ! save time : 0.11942148208618164 No data in photo_id : 1383950585 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.05305671691894531 time for calcul the mask position with numpy : 0.14584946632385254 nb_pixel_total : 2042133 time to create 1 rle with new method : 0.16342806816101074 time for calcul the mask position with numpy : 0.006479501724243164 nb_pixel_total : 14892 time to create 1 rle with old method : 0.0164945125579834 time for calcul the mask position with numpy : 0.006005048751831055 nb_pixel_total : 9237 time to create 1 rle with old method : 0.010196924209594727 time for calcul the mask position with numpy : 0.006206989288330078 nb_pixel_total : 7338 time to create 1 rle with old method : 0.008278846740722656 create new chi : 0.373485803604126 time to delete rle : 0.00027632713317871094 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1714 TO DO : save crop sub photo not yet done ! save time : 0.1537768840789795 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.07555341720581055 time for calcul the mask position with numpy : 0.03281259536743164 nb_pixel_total : 1040851 time to create 1 rle with new method : 0.10428094863891602 time for calcul the mask position with numpy : 0.0061147212982177734 nb_pixel_total : 6612 time to create 1 rle with old method : 0.0073680877685546875 time for calcul the mask position with numpy : 0.012930154800415039 nb_pixel_total : 1016387 time to create 1 rle with new method : 0.166093111038208 time for calcul the mask position with numpy : 0.006405353546142578 nb_pixel_total : 9750 time to create 1 rle with old method : 0.011060953140258789 create new chi : 0.36352992057800293 time to delete rle : 0.00064849853515625 batch 1 Loaded 8 chid ids of type : 3594 +++++++++Number RLEs to save : 4449 TO DO : save crop sub photo not yet done ! save time : 0.31214213371276855 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.07996845245361328 time for calcul the mask position with numpy : 0.1289963722229004 nb_pixel_total : 2041452 time to create 1 rle with new method : 0.07724380493164062 time for calcul the mask position with numpy : 0.005975246429443359 nb_pixel_total : 4174 time to create 1 rle with old method : 0.004538536071777344 time for calcul the mask position with numpy : 0.005932807922363281 nb_pixel_total : 3580 time to create 1 rle with old method : 0.0037882328033447266 time for calcul the mask position with numpy : 0.005956411361694336 nb_pixel_total : 11381 time to create 1 rle with old method : 0.012589693069458008 time for calcul the mask position with numpy : 0.00672149658203125 nb_pixel_total : 13013 time to create 1 rle with old method : 0.014690876007080078 create new chi : 0.2771286964416504 time to delete rle : 0.00033020973205566406 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 1872 TO DO : save crop sub photo not yet done ! save time : 0.13759112358093262 No data in photo_id : 1383950552 nb_obj : 7 nb_hashtags : 1 time to prepare the origin masks : 0.14886093139648438 time for calcul the mask position with numpy : 0.059943437576293945 nb_pixel_total : 2042557 time to create 1 rle with new method : 0.18430352210998535 time for calcul the mask position with numpy : 0.006256580352783203 nb_pixel_total : 2537 time to create 1 rle with old method : 0.002841472625732422 time for calcul the mask position with numpy : 0.006532192230224609 nb_pixel_total : 9037 time to create 1 rle with old method : 0.010288000106811523 time for calcul the mask position with numpy : 0.006240367889404297 nb_pixel_total : 2047 time to create 1 rle with old method : 0.0022749900817871094 time for calcul the mask position with numpy : 0.0062198638916015625 nb_pixel_total : 8993 time to create 1 rle with old method : 0.012393951416015625 time for calcul the mask position with numpy : 0.0064203739166259766 nb_pixel_total : 3240 time to create 1 rle with old method : 0.003396749496459961 time for calcul the mask position with numpy : 0.006491184234619141 nb_pixel_total : 2293 time to create 1 rle with old method : 0.0026450157165527344 time for calcul the mask position with numpy : 0.006395578384399414 nb_pixel_total : 2896 time to create 1 rle with old method : 0.0031914710998535156 create new chi : 0.33658337593078613 time to delete rle : 0.0003731250762939453 batch 1 Loaded 15 chid ids of type : 3594 +++++++Number RLEs to save : 2182 TO DO : save crop sub photo not yet done ! save time : 0.1764986515045166 map_output_result : {1383950821: (0.0, 'Should be the crop_list due to order', 0), 1383950819: (0.0, 'Should be the crop_list due to order', 0), 1383950816: (0.0, 'Should be the crop_list due to order', 0), 1383950811: (0.0, 'Should be the crop_list due to order', 0), 1383950589: (0.0, 'Should be the crop_list due to order', 0), 1383950588: (0.0, 'Should be the crop_list due to order', 0), 1383950585: (0.0, 'Should be the crop_list due to order', 0.0), 1383950582: (0.0, 'Should be the crop_list due to order', 0), 1383950578: (0.0, 'Should be the crop_list due to order', 0), 1383950576: (0.0, 'Should be the crop_list due to order', 0), 1383950552: (0.0, 'Should be the crop_list due to order', 0.0), 1383950547: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1383950821, 1383950819, 1383950816, 1383950811, 1383950589, 1383950588, 1383950585, 1383950582, 1383950578, 1383950576, 1383950552, 1383950547] Looping around the photos to save general results len do output : 12 /1383950821.Didn't retrieve data . /1383950819.Didn't retrieve data . /1383950816.Didn't retrieve data . /1383950811.Didn't retrieve data . /1383950589.Didn't retrieve data . /1383950588.Didn't retrieve data . /1383950585.Didn't retrieve data . /1383950582.Didn't retrieve data . /1383950578.Didn't retrieve data . /1383950576.Didn't retrieve data . /1383950552.Didn't retrieve data . /1383950547.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950821', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950819', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950816', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950811', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950589', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950588', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950585', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950582', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950578', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950576', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950552', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950547', None, None, None, None, None, '3729397') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 36 time used for this insertion : 0.014035701751708984 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.147761583328247 time spend to save output : 0.014461040496826172 total time spend for step 3 : 8.162222623825073 step4:ventilate_hashtags_in_portfolio Tue Sep 16 18:01:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26902191 get user id for portfolio 26902191 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26902191 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','papier','flou','mal_croppe','background','metal','pehd','environnement','carton','pet_clair','autre')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26902191 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','papier','flou','mal_croppe','background','metal','pehd','environnement','carton','pet_clair','autre')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26902191 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','papier','flou','mal_croppe','background','metal','pehd','environnement','carton','pet_clair','autre')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26902333,26902334,26902335,26902336,26902337,26902338,26902339,26902340,26902341,26902342,26902343?tags=pet_fonce,papier,flou,mal_croppe,background,metal,pehd,environnement,carton,pet_clair,autre Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1383950821, 1383950819, 1383950816, 1383950811, 1383950589, 1383950588, 1383950585, 1383950582, 1383950578, 1383950576, 1383950552, 1383950547] Looping around the photos to save general results len do output : 1 /26902191. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950821', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950819', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950816', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950811', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950589', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950588', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950585', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950582', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950578', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950576', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950552', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950547', None, None, None, None, None, '3729397') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.01652812957763672 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6769349575042725 time spend to save output : 0.016790151596069336 total time spend for step 4 : 1.6937251091003418 step5:final Tue Sep 16 18:01:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1383950821: ('0.14204439139660488',), 1383950819: ('0.14204439139660488',), 1383950816: ('0.14204439139660488',), 1383950811: ('0.14204439139660488',), 1383950589: ('0.14204439139660488',), 1383950588: ('0.14204439139660488',), 1383950585: ('0.14204439139660488',), 1383950582: ('0.14204439139660488',), 1383950578: ('0.14204439139660488',), 1383950576: ('0.14204439139660488',), 1383950552: ('0.14204439139660488',), 1383950547: ('0.14204439139660488',)} new output for save of step final : {1383950821: ('0.14204439139660488',), 1383950819: ('0.14204439139660488',), 1383950816: ('0.14204439139660488',), 1383950811: ('0.14204439139660488',), 1383950589: ('0.14204439139660488',), 1383950588: ('0.14204439139660488',), 1383950585: ('0.14204439139660488',), 1383950582: ('0.14204439139660488',), 1383950578: ('0.14204439139660488',), 1383950576: ('0.14204439139660488',), 1383950552: ('0.14204439139660488',), 1383950547: ('0.14204439139660488',)} [1383950821, 1383950819, 1383950816, 1383950811, 1383950589, 1383950588, 1383950585, 1383950582, 1383950578, 1383950576, 1383950552, 1383950547] Looping around the photos to save general results len do output : 12 /1383950821.Didn't retrieve data . /1383950819.Didn't retrieve data . /1383950816.Didn't retrieve data . /1383950811.Didn't retrieve data . /1383950589.Didn't retrieve data . /1383950588.Didn't retrieve data . /1383950585.Didn't retrieve data . /1383950582.Didn't retrieve data . /1383950578.Didn't retrieve data . /1383950576.Didn't retrieve data . /1383950552.Didn't retrieve data . /1383950547.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950821', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950819', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950816', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950811', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950589', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950588', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950585', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950582', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950578', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950576', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950552', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950547', None, None, None, None, None, '3729397') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 36 time used for this insertion : 0.016878128051757812 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12321162223815918 time spend to save output : 0.017698049545288086 total time spend for step 5 : 0.14090967178344727 step6:blur_detection Tue Sep 16 18:01:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94.jpg resize: (1080, 1920) 1383950821 -0.3743610987735359 treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c.jpg resize: (1080, 1920) 1383950819 0.8705918722911614 treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9.jpg resize: (1080, 1920) 1383950816 0.28661454861664104 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7.jpg resize: (1080, 1920) 1383950811 -0.9932720133946855 treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d.jpg resize: (1080, 1920) 1383950589 -0.3546015623895782 treat image : temp/1758038426_3332805_1383950588_48c14aaa548267555ca905de3aeb5ec7.jpg resize: (1080, 1920) 1383950588 1.3655391186512644 treat image : temp/1758038426_3332805_1383950585_8785d94f3765525e289223ce98a7aae3.jpg resize: (1080, 1920) 1383950585 -0.21591879515944087 treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59.jpg resize: (1080, 1920) 1383950582 0.07038207025616904 treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7.jpg resize: (1080, 1920) 1383950578 1.6937248478565898 treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708.jpg resize: (1080, 1920) 1383950576 -0.641596959815593 treat image : temp/1758038426_3332805_1383950552_3ee930a0d5cf82d58d0050d55b210801.jpg resize: (1080, 1920) 1383950552 -0.5434690351746162 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661.jpg resize: (1080, 1920) 1383950547 -0.21605381252376402 treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738327_0.png resize: (139, 99) 1383955402 -2.135983282335857 treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738330_0.png resize: (131, 99) 1383955403 -1.4860762178883127 treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738331_0.png resize: (150, 178) 1383955405 -0.7091173152113827 treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738332_0.png resize: (162, 153) 1383955408 0.5379439055691245 treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738334_0.png resize: (81, 97) 1383955409 -0.24947846132583967 treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738335_0.png resize: (128, 127) 1383955411 -1.7350502239926309 treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9_rle_crop_3959738336_0.png resize: (193, 205) 1383955414 0.4752716956695763 treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9_rle_crop_3959738337_0.png resize: (76, 88) 1383955415 -2.00437771080595 treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9_rle_crop_3959738338_0.png resize: (53, 176) 1383955417 0.4107031934624287 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738339_0.png resize: (117, 115) 1383955420 -1.955582249744502 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738340_0.png resize: (139, 138) 1383955421 -0.8503666586547087 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738342_0.png resize: (68, 81) 1383955423 -1.5541010749008417 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738343_0.png resize: (60, 95) 1383955426 0.3929624683023811 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738344_0.png resize: (67, 117) 1383955427 -2.4316595576166176 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738346_0.png resize: (107, 112) 1383955429 0.08773664915188281 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738348_0.png resize: (69, 137) 1383955432 -1.0476650312322295 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738349_0.png resize: (114, 76) 1383955433 -3.079602604291686 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738351_0.png resize: (42, 95) 1383955435 -0.6153041367320882 treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738352_0.png resize: (164, 200) 1383955438 -2.016309618068437 treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738353_0.png resize: (44, 90) 1383955440 3.355280154209411 treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738355_0.png resize: (111, 151) 1383955442 -1.7180720682432669 treat image : temp/1758038426_3332805_1383950588_48c14aaa548267555ca905de3aeb5ec7_rle_crop_3959738356_0.png resize: (140, 167) 1383955444 -0.14746357191770498 treat image : temp/1758038426_3332805_1383950588_48c14aaa548267555ca905de3aeb5ec7_rle_crop_3959738357_0.png resize: (65, 104) 1383955446 -1.1291737224935454 treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59_rle_crop_3959738358_0.png resize: (91, 112) 1383955448 0.755497636453723 treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59_rle_crop_3959738359_0.png resize: (98, 125) 1383955450 -1.3972698467077023 treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59_rle_crop_3959738360_0.png resize: (122, 139) 1383955452 2.244490518565407 treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738361_0.png resize: (177, 113) 1383955454 -1.9532545758911704 treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738363_0.png resize: (121, 124) 1383955456 -0.6523311397194995 treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738364_0.png resize: (66, 87) 1383955458 -1.5642196999610403 treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738365_0.png resize: (118, 185) 1383955460 -1.057027836107554 treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738367_0.png resize: (66, 83) 1383955462 -1.1906986768147014 treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738368_0.png resize: (102, 69) 1383955464 -1.8201414194587793 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738369_0.png resize: (65, 49) 1383955466 20.0 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738370_0.png resize: (55, 70) 1383955468 -2.051925456454425 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738371_0.png resize: (58, 63) 1383955470 7.443169762053376 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738372_0.png resize: (112, 127) 1383955472 -2.3023190512935585 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738373_0.png resize: (75, 35) 1383955474 0.8179454106810295 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738374_0.png resize: (123, 111) 1383955476 -1.7108615870243162 treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738375_0.png resize: (56, 67) 1383955478 -2.5775720619322833 treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738366_0.png resize: (107, 141) 1383955497 -3.0763101085017235 treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738326_0.png resize: (43, 89) 1383955511 0.3157112071252384 treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738328_0.png resize: (259, 257) 1383955862 -1.3028242737669136 treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738329_0.png resize: (170, 138) 1383955865 -0.571621443577329 treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738333_0.png resize: (521, 337) 1383955867 0.44898221337432015 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738341_0.png resize: (39, 144) 1383955868 -0.4435191385106976 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738345_0.png resize: (540, 290) 1383955871 -0.4415672359410265 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738347_0.png resize: (447, 401) 1383955873 -1.2524937809275098 treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738350_0.png resize: (990, 1046) 1383955874 -0.1871983663354338 treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738354_0.png resize: (981, 1380) 1383955877 -1.2839606943471253 treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738362_0.png resize: (1006, 1192) 1383955879 -1.1440944230579952 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 62 time used for this insertion : 0.018184423446655273 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 62 time used for this insertion : 0.015555143356323242 save missing photos in datou_result : time spend for datou_step_exec : 11.224158763885498 time spend to save output : 0.03856062889099121 total time spend for step 6 : 11.26271939277649 step7:brightness Tue Sep 16 18:01:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94.jpg treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c.jpg treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9.jpg treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7.jpg treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d.jpg treat image : temp/1758038426_3332805_1383950588_48c14aaa548267555ca905de3aeb5ec7.jpg treat image : temp/1758038426_3332805_1383950585_8785d94f3765525e289223ce98a7aae3.jpg treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59.jpg treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7.jpg treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708.jpg treat image : temp/1758038426_3332805_1383950552_3ee930a0d5cf82d58d0050d55b210801.jpg treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661.jpg treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738327_0.png treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738330_0.png treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738331_0.png treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738332_0.png treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738334_0.png treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738335_0.png treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9_rle_crop_3959738336_0.png treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9_rle_crop_3959738337_0.png treat image : temp/1758038426_3332805_1383950816_e682f6edce85308ec926e49fb1eec8f9_rle_crop_3959738338_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738339_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738340_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738342_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738343_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738344_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738346_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738348_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738349_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738351_0.png treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738352_0.png treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738353_0.png treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738355_0.png treat image : temp/1758038426_3332805_1383950588_48c14aaa548267555ca905de3aeb5ec7_rle_crop_3959738356_0.png treat image : temp/1758038426_3332805_1383950588_48c14aaa548267555ca905de3aeb5ec7_rle_crop_3959738357_0.png treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59_rle_crop_3959738358_0.png treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59_rle_crop_3959738359_0.png treat image : temp/1758038426_3332805_1383950582_d307fc908554e58b3ba1de4769db3d59_rle_crop_3959738360_0.png treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738361_0.png treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738363_0.png treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738364_0.png treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738365_0.png treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738367_0.png treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738368_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738369_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738370_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738371_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738372_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738373_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738374_0.png treat image : temp/1758038426_3332805_1383950547_8ae4e977cf35ceaa29d2ceee99364661_rle_crop_3959738375_0.png treat image : temp/1758038426_3332805_1383950576_e8b297e4dde9be925689696275398708_rle_crop_3959738366_0.png treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738326_0.png treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738328_0.png treat image : temp/1758038426_3332805_1383950821_30d455839496384cdc2e52a101c0fe94_rle_crop_3959738329_0.png treat image : temp/1758038426_3332805_1383950819_93c743114998613f9b55b03835d1c09c_rle_crop_3959738333_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738341_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738345_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738347_0.png treat image : temp/1758038426_3332805_1383950811_5f6af48b30e6ef29abcb5f066918a3e7_rle_crop_3959738350_0.png treat image : temp/1758038426_3332805_1383950589_75c8f52ba7d4e03b5134eb0013c56f7d_rle_crop_3959738354_0.png treat image : temp/1758038426_3332805_1383950578_abc34067ecd2f47f34ef58a0ef46bfa7_rle_crop_3959738362_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 62 time used for this insertion : 0.015863418579101562 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 62 time used for this insertion : 0.016094207763671875 save missing photos in datou_result : time spend for datou_step_exec : 3.2567574977874756 time spend to save output : 0.03666234016418457 total time spend for step 7 : 3.29341983795166 step8:velours_tree Tue Sep 16 18:01:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.10421466827392578 time spend to save output : 5.412101745605469e-05 total time spend for step 8 : 0.10426878929138184 step9:send_mail_cod Tue Sep 16 18:01:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26902191_16-09-2025_18_01_53.pdf 26902333 imagette269023331758038513 26902334 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette269023341758038513 26902335 imagette269023351758038515 26902336 imagette269023361758038515 26902337 imagette269023371758038515 26902338 change filename to text .imagette269023381758038515 26902339 imagette269023391758038515 26902341 change filename to text .imagette269023411758038515 26902342 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette269023421758038515 26902343 imagette269023431758038516 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26902191 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26902333,26902334,26902335,26902336,26902337,26902338,26902339,26902340,26902341,26902342,26902343?tags=pet_fonce,papier,flou,mal_croppe,background,metal,pehd,environnement,carton,pet_clair,autre args[1383950821] : ((1383950821, -0.3743610987735359, 492688767), (1383950821, 0.7372833622971288, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950819] : ((1383950819, 0.8705918722911614, 492688767), (1383950819, 0.5934551252659351, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950816] : ((1383950816, 0.28661454861664104, 492688767), (1383950816, 0.2413898986197363, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950811] : ((1383950811, -0.9932720133946855, 492688767), (1383950811, 0.7015608368972419, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950589] : ((1383950589, -0.3546015623895782, 492688767), (1383950589, 0.6887369814373923, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950588] : ((1383950588, 1.3655391186512644, 492688767), (1383950588, 0.6785089857194471, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950585] : ((1383950585, -0.21591879515944087, 492688767), (1383950585, 0.6207134523316448, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950582] : ((1383950582, 0.07038207025616904, 492688767), (1383950582, 0.5498618487641439, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950578] : ((1383950578, 1.6937248478565898, 492688767), (1383950578, 0.4284686833765842, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950576] : ((1383950576, -0.641596959815593, 492688767), (1383950576, 0.6589499072691966, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950552] : ((1383950552, -0.5434690351746162, 492688767), (1383950552, 0.8573135250962663, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com args[1383950547] : ((1383950547, -0.21605381252376402, 492688767), (1383950547, 0.7220365977466894, 2107752395), '0.14204439139660488') We are sending mail with results at report@fotonower.com refus_total : 0.14204439139660488 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26902191 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26902191_16-09-2025_18_01_53.pdf results_Auto_P26902191_16-09-2025_18_01_53.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26902191_16-09-2025_18_01_53.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26902191','results_Auto_P26902191_16-09-2025_18_01_53.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26902191_16-09-2025_18_01_53.pdf','pdf','','0.48','0.14204439139660488') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26902191

https://www.fotonower.com/image?json=false&list_photos_id=1383950821
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950819
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950816
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950811
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950589
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950588
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.3655391186512644)
https://www.fotonower.com/image?json=false&list_photos_id=1383950585
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950582
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950578
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.6937248478565898)
https://www.fotonower.com/image?json=false&list_photos_id=1383950576
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950552
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383950547
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 14.20%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/26902334?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/26902338?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26902341?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26902342?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26902191_16-09-2025_18_01_53.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/26902333,26902334,26902335,26902336,26902337,26902338,26902339,26902340,26902341,26902342,26902343?tags=pet_fonce,papier,flou,mal_croppe,background,metal,pehd,environnement,carton,pet_clair,autre.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 16 Sep 2025 16:01:58 GMT Content-Length: 0 Connection: close X-Message-Id: TJEjeSurQhSRPeAPWoIzWg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1383950821, 1383950819, 1383950816, 1383950811, 1383950589, 1383950588, 1383950585, 1383950582, 1383950578, 1383950576, 1383950552, 1383950547] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950821', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950819', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950816', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950811', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950589', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950588', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950585', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950582', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950578', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950576', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950552', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950547', None, None, None, None, None, '3729397') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.016351938247680664 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.255965948104858 time spend to save output : 0.016704797744750977 total time spend for step 9 : 4.272670745849609 step10:split_time_score Tue Sep 16 18:01:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('17', 12),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 16092025 26902191 Nombre de photos uploadées : 12 / 23040 (0%) 16092025 26902191 Nombre de photos taguées (types de déchets): 0 / 12 (0%) 16092025 26902191 Nombre de photos taguées (volume) : 0 / 12 (0%) elapsed_time : load_data_split_time_score 4.76837158203125e-06 elapsed_time : order_list_meta_photo_and_scores 1.6689300537109375e-05 ???????????? elapsed_time : fill_and_build_computed_from_old_data 0.0008652210235595703 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.2057018280029297 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.1122128041031227 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26882766_16-09-2025_09_52_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26882766 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26882766 AND mptpi.`type`=3594 To do Qualite : 0.1705833654835391 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26882768_16-09-2025_09_41_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26882768 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26882768 AND mptpi.`type`=3594 To do Qualite : 0.05712583188657407 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26882771_16-09-2025_10_31_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26882771 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26882771 AND mptpi.`type`=3594 To do Qualite : 0.0553948408253684 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26882772_16-09-2025_10_53_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26882772 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26882772 AND mptpi.`type`=3594 To do Qualite : 0.03425365770417854 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26891609_16-09-2025_15_22_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26891609 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26891609 AND mptpi.`type`=3594 To do Qualite : 0.07995926568930034 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26891611_16-09-2025_13_53_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26891611 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26891611 AND mptpi.`type`=3594 To do Qualite : 0.09510918209876545 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26888808_16-09-2025_11_41_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26888808 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26888808 AND mptpi.`type`=3594 To do Qualite : 0.1363156741021324 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26890190_16-09-2025_12_31_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26890190 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26890190 AND mptpi.`type`=3594 To do Qualite : 0.06626128127755732 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26890528_16-09-2025_12_42_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26890528 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26890528 AND mptpi.`type`=3594 To do Qualite : 0.2100799961419753 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26891614_16-09-2025_13_33_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26891614 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26891614 AND mptpi.`type`=3594 To do Qualite : 0.09143330693632225 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26894974_16-09-2025_15_12_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26894974 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26894974 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26902185 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26902187 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26902189 order by id desc limit 1 Qualite : 0.14204439139660488 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26902191_16-09-2025_18_01_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26902191 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26902191 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'16092025': {'nb_upload': 12, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1383950821, 1383950819, 1383950816, 1383950811, 1383950589, 1383950588, 1383950585, 1383950582, 1383950578, 1383950576, 1383950552, 1383950547] Looping around the photos to save general results len do output : 1 /26902191Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950821', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950819', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950816', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950811', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950589', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950588', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950585', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950582', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950578', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950576', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950552', None, None, None, None, None, '3729397') ('3318', None, None, None, None, None, None, None, '3729397') ('3318', '26902191', '1383950547', None, None, None, None, None, '3729397') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.02390766143798828 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.106723308563232 time spend to save output : 0.024358272552490234 total time spend for step 10 : 4.131081581115723 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 12 set_done_treatment 50.94user 21.48system 1:37.93elapsed 73%CPU (0avgtext+0avgdata 2901500maxresident)k 504224inputs+36064outputs (68major+1845334minor)pagefaults 0swaps