python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1251003 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3694309'] with mtr_portfolio_ids : ['26751062'] and first list_photo_ids : [] new path : /proc/1251003/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 1.1658530235290527 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu Sep 11 10:00:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10594 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-11 10:00:32.988874: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-11 10:00:33.016596: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-11 10:00:33.018785: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f88b4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-11 10:00:33.018846: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-11 10:00:33.022774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-11 10:00:33.190275: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x509e060 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-11 10:00:33.190335: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-11 10:00:33.191728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-11 10:00:33.192152: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-11 10:00:33.195367: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-11 10:00:33.198614: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-11 10:00:33.199116: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-11 10:00:33.202135: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-11 10:00:33.203170: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-11 10:00:33.207609: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-11 10:00:33.209067: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-11 10:00:33.209140: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-11 10:00:33.209867: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-11 10:00:33.209881: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-11 10:00:33.209890: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-11 10:00:33.211167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9813 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-11 10:00:33.476065: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-11 10:00:33.476184: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-11 10:00:33.476205: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-11 10:00:33.476224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-11 10:00:33.476242: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-11 10:00:33.476259: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-11 10:00:33.476276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-11 10:00:33.476294: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-11 10:00:33.477858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-11 10:00:33.479289: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-11 10:00:33.479325: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-11 10:00:33.479343: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-11 10:00:33.479360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-11 10:00:33.479377: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-11 10:00:33.479394: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-11 10:00:33.479411: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-11 10:00:33.479427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-11 10:00:33.480984: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-11 10:00:33.481026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-11 10:00:33.481037: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-11 10:00:33.481046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-11 10:00:33.482369: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9813 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-11 10:00:41.354200: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-11 10:00:41.537798: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 42.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 49.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 36.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 Detection mask done ! Trying to reset tf kernel 1251769 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4503 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9698 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0004208087921142578 nb_pixel_total : 13594 time to create 1 rle with old method : 0.015443086624145508 length of segment : 143 time for calcul the mask position with numpy : 0.0006604194641113281 nb_pixel_total : 21952 time to create 1 rle with old method : 0.033249616622924805 length of segment : 188 time for calcul the mask position with numpy : 0.0003898143768310547 nb_pixel_total : 21648 time to create 1 rle with old method : 0.02567887306213379 length of segment : 238 time for calcul the mask position with numpy : 0.00038886070251464844 nb_pixel_total : 23493 time to create 1 rle with old method : 0.02662968635559082 length of segment : 276 time for calcul the mask position with numpy : 0.00011777877807617188 nb_pixel_total : 6293 time to create 1 rle with old method : 0.007651805877685547 length of segment : 72 time for calcul the mask position with numpy : 0.0002186298370361328 nb_pixel_total : 10437 time to create 1 rle with old method : 0.012034177780151367 length of segment : 181 time for calcul the mask position with numpy : 0.0004398822784423828 nb_pixel_total : 15730 time to create 1 rle with old method : 0.018784523010253906 length of segment : 224 time for calcul the mask position with numpy : 0.00021266937255859375 nb_pixel_total : 9734 time to create 1 rle with old method : 0.011675596237182617 length of segment : 164 time for calcul the mask position with numpy : 0.00014448165893554688 nb_pixel_total : 4260 time to create 1 rle with old method : 0.00533294677734375 length of segment : 164 time for calcul the mask position with numpy : 0.0001990795135498047 nb_pixel_total : 10585 time to create 1 rle with old method : 0.012685298919677734 length of segment : 159 time for calcul the mask position with numpy : 0.000102996826171875 nb_pixel_total : 4908 time to create 1 rle with old method : 0.005985260009765625 length of segment : 69 time for calcul the mask position with numpy : 0.00015354156494140625 nb_pixel_total : 9946 time to create 1 rle with old method : 0.011491060256958008 length of segment : 152 time for calcul the mask position with numpy : 0.0002760887145996094 nb_pixel_total : 9500 time to create 1 rle with old method : 0.011350631713867188 length of segment : 182 time for calcul the mask position with numpy : 0.0003414154052734375 nb_pixel_total : 17307 time to create 1 rle with old method : 0.020739078521728516 length of segment : 162 time for calcul the mask position with numpy : 0.0005295276641845703 nb_pixel_total : 32359 time to create 1 rle with old method : 0.03824114799499512 length of segment : 179 time for calcul the mask position with numpy : 0.0002613067626953125 nb_pixel_total : 15377 time to create 1 rle with old method : 0.01840066909790039 length of segment : 124 time for calcul the mask position with numpy : 0.000293731689453125 nb_pixel_total : 12843 time to create 1 rle with old method : 0.021756887435913086 length of segment : 116 time for calcul the mask position with numpy : 0.0004363059997558594 nb_pixel_total : 26091 time to create 1 rle with old method : 0.030997037887573242 length of segment : 209 time for calcul the mask position with numpy : 0.00019621849060058594 nb_pixel_total : 7718 time to create 1 rle with old method : 0.009277582168579102 length of segment : 146 time for calcul the mask position with numpy : 0.00011157989501953125 nb_pixel_total : 5045 time to create 1 rle with old method : 0.006028652191162109 length of segment : 87 time for calcul the mask position with numpy : 0.00025582313537597656 nb_pixel_total : 14942 time to create 1 rle with old method : 0.017559289932250977 length of segment : 169 time for calcul the mask position with numpy : 0.00011658668518066406 nb_pixel_total : 4866 time to create 1 rle with old method : 0.006056785583496094 length of segment : 75 time for calcul the mask position with numpy : 0.00010991096496582031 nb_pixel_total : 5578 time to create 1 rle with old method : 0.0069425106048583984 length of segment : 73 time for calcul the mask position with numpy : 0.0002608299255371094 nb_pixel_total : 9175 time to create 1 rle with old method : 0.010843992233276367 length of segment : 183 time for calcul the mask position with numpy : 0.00010728836059570312 nb_pixel_total : 3663 time to create 1 rle with old method : 0.004683971405029297 length of segment : 46 time for calcul the mask position with numpy : 0.008931875228881836 nb_pixel_total : 528828 time to create 1 rle with new method : 0.01986527442932129 length of segment : 1145 time for calcul the mask position with numpy : 0.00022673606872558594 nb_pixel_total : 12195 time to create 1 rle with old method : 0.017721176147460938 length of segment : 163 time for calcul the mask position with numpy : 0.0004203319549560547 nb_pixel_total : 21908 time to create 1 rle with old method : 0.03601574897766113 length of segment : 179 time for calcul the mask position with numpy : 0.00022554397583007812 nb_pixel_total : 10113 time to create 1 rle with old method : 0.012180566787719727 length of segment : 124 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 9141 time to create 1 rle with old method : 0.011096954345703125 length of segment : 129 time for calcul the mask position with numpy : 0.00013375282287597656 nb_pixel_total : 5374 time to create 1 rle with old method : 0.006601810455322266 length of segment : 90 time for calcul the mask position with numpy : 0.0001575946807861328 nb_pixel_total : 5727 time to create 1 rle with old method : 0.0071964263916015625 length of segment : 96 time for calcul the mask position with numpy : 0.0001976490020751953 nb_pixel_total : 10079 time to create 1 rle with old method : 0.012122154235839844 length of segment : 145 time for calcul the mask position with numpy : 0.00022149085998535156 nb_pixel_total : 9259 time to create 1 rle with old method : 0.010994911193847656 length of segment : 172 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 3359 time to create 1 rle with old method : 0.008880615234375 length of segment : 37 time for calcul the mask position with numpy : 0.0003650188446044922 nb_pixel_total : 7451 time to create 1 rle with old method : 0.009077072143554688 length of segment : 100 time for calcul the mask position with numpy : 0.00044727325439453125 nb_pixel_total : 12519 time to create 1 rle with old method : 0.015021324157714844 length of segment : 110 time for calcul the mask position with numpy : 0.0008685588836669922 nb_pixel_total : 32341 time to create 1 rle with old method : 0.03806591033935547 length of segment : 301 time for calcul the mask position with numpy : 0.0003960132598876953 nb_pixel_total : 10481 time to create 1 rle with old method : 0.012528181076049805 length of segment : 134 time for calcul the mask position with numpy : 0.0003540515899658203 nb_pixel_total : 10410 time to create 1 rle with old method : 0.012297630310058594 length of segment : 105 time for calcul the mask position with numpy : 0.0002856254577636719 nb_pixel_total : 10259 time to create 1 rle with old method : 0.011771917343139648 length of segment : 110 time for calcul the mask position with numpy : 0.0002815723419189453 nb_pixel_total : 9046 time to create 1 rle with old method : 0.010812044143676758 length of segment : 78 time for calcul the mask position with numpy : 0.00019121170043945312 nb_pixel_total : 3255 time to create 1 rle with old method : 0.003989458084106445 length of segment : 77 time for calcul the mask position with numpy : 0.0007417201995849609 nb_pixel_total : 19646 time to create 1 rle with old method : 0.023554086685180664 length of segment : 291 time for calcul the mask position with numpy : 0.000469207763671875 nb_pixel_total : 18284 time to create 1 rle with old method : 0.02170705795288086 length of segment : 107 time for calcul the mask position with numpy : 0.00017523765563964844 nb_pixel_total : 1486 time to create 1 rle with old method : 0.0019652843475341797 length of segment : 70 time for calcul the mask position with numpy : 0.00019288063049316406 nb_pixel_total : 3247 time to create 1 rle with old method : 0.004156827926635742 length of segment : 95 time for calcul the mask position with numpy : 0.0003943443298339844 nb_pixel_total : 10447 time to create 1 rle with old method : 0.012569665908813477 length of segment : 94 time for calcul the mask position with numpy : 0.00019669532775878906 nb_pixel_total : 7642 time to create 1 rle with old method : 0.009755134582519531 length of segment : 101 time for calcul the mask position with numpy : 0.0004341602325439453 nb_pixel_total : 9204 time to create 1 rle with old method : 0.011625528335571289 length of segment : 145 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 1751 time to create 1 rle with old method : 0.0021932125091552734 length of segment : 41 time spent for convertir_results : 2.203819513320923 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 51 chid ids of type : 3594 Number RLEs to save : 8020 save missing photos in datou_result : time spend for datou_step_exec : 25.95495367050171 time spend to save output : 0.7044646739959717 total time spend for step 1 : 26.65941834449768 step2:crop_condition Thu Sep 11 10:00:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 38 About to insert : list_path_to_insert length 38 new photo from crops ! About to upload 38 photos upload in portfolio : 3736932 init cache_photo without model_param we have 38 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757577659_1251003 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 38 photos in the portfolio 3736932 time of upload the photos Elapsed time : 10.901172876358032 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757577670_1251003 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6141231060028076 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3736932 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757577674_1251003 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 12 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.4743878841400146 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1383206553, 1383206500, 1383206445, 1383206398, 1383206057, 1383206003, 1383205955, 1383205908, 1383205860] Looping around the photos to save general results len do output : 51 /1383242504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242556Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242605Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1383242616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206553', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206500', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206445', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206398', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206057', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206003', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205955', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205908', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205860', None, None, None, None, None, '3694309') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 162 time used for this insertion : 0.11609411239624023 save_final save missing photos in datou_result : time spend for datou_step_exec : 20.621540784835815 time spend to save output : 0.11821961402893066 total time spend for step 2 : 20.739760398864746 step3:rle_unique_nms_with_priority Thu Sep 11 10:01:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 51 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.3306560516357422 time for calcul the mask position with numpy : 0.07450103759765625 nb_pixel_total : 2016406 time to create 1 rle with new method : 0.10203146934509277 time for calcul the mask position with numpy : 0.0077936649322509766 nb_pixel_total : 21648 time to create 1 rle with old method : 0.024927616119384766 time for calcul the mask position with numpy : 0.007309436798095703 nb_pixel_total : 21952 time to create 1 rle with old method : 0.024204492568969727 time for calcul the mask position with numpy : 0.0063664913177490234 nb_pixel_total : 13594 time to create 1 rle with old method : 0.01510167121887207 create new chi : 0.273665189743042 time to delete rle : 0.02798748016357422 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 2218 TO DO : save crop sub photo not yet done ! save time : 0.748816728591919 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.2916693687438965 time for calcul the mask position with numpy : 0.07944107055664062 nb_pixel_total : 2020991 time to create 1 rle with new method : 0.35340285301208496 time for calcul the mask position with numpy : 0.0064334869384765625 nb_pixel_total : 15730 time to create 1 rle with old method : 0.017442941665649414 time for calcul the mask position with numpy : 0.006741762161254883 nb_pixel_total : 10437 time to create 1 rle with old method : 0.01157522201538086 time for calcul the mask position with numpy : 0.005974531173706055 nb_pixel_total : 2949 time to create 1 rle with old method : 0.003542661666870117 time for calcul the mask position with numpy : 0.006547689437866211 nb_pixel_total : 23493 time to create 1 rle with old method : 0.02612900733947754 create new chi : 0.5273387432098389 time to delete rle : 0.00038933753967285156 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 2521 TO DO : save crop sub photo not yet done ! save time : 0.21568059921264648 nb_obj : 7 nb_hashtags : 2 time to prepare the origin masks : 0.32977795600891113 time for calcul the mask position with numpy : 0.08027338981628418 nb_pixel_total : 2007360 time to create 1 rle with new method : 0.23655247688293457 time for calcul the mask position with numpy : 0.009290695190429688 nb_pixel_total : 17307 time to create 1 rle with old method : 0.02053070068359375 time for calcul the mask position with numpy : 0.008099794387817383 nb_pixel_total : 9500 time to create 1 rle with old method : 0.01055288314819336 time for calcul the mask position with numpy : 0.007715940475463867 nb_pixel_total : 9946 time to create 1 rle with old method : 0.01124715805053711 time for calcul the mask position with numpy : 0.008397102355957031 nb_pixel_total : 4908 time to create 1 rle with old method : 0.0062296390533447266 time for calcul the mask position with numpy : 0.010239362716674805 nb_pixel_total : 10585 time to create 1 rle with old method : 0.015111684799194336 time for calcul the mask position with numpy : 0.00890350341796875 nb_pixel_total : 4260 time to create 1 rle with old method : 0.004781246185302734 time for calcul the mask position with numpy : 0.007564544677734375 nb_pixel_total : 9734 time to create 1 rle with old method : 0.011111259460449219 create new chi : 0.46845507621765137 time to delete rle : 0.0007424354553222656 batch 1 Loaded 15 chid ids of type : 3594 +++++++Number RLEs to save : 3184 TO DO : save crop sub photo not yet done ! save time : 0.3585209846496582 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 0.3008542060852051 time for calcul the mask position with numpy : 0.2200608253479004 nb_pixel_total : 1983535 time to create 1 rle with new method : 0.10833978652954102 time for calcul the mask position with numpy : 0.007950544357299805 nb_pixel_total : 1238 time to create 1 rle with old method : 0.0016019344329833984 time for calcul the mask position with numpy : 0.008439779281616211 nb_pixel_total : 14942 time to create 1 rle with old method : 0.017208337783813477 time for calcul the mask position with numpy : 0.007873296737670898 nb_pixel_total : 5045 time to create 1 rle with old method : 0.005846977233886719 time for calcul the mask position with numpy : 0.007995128631591797 nb_pixel_total : 7718 time to create 1 rle with old method : 0.008806943893432617 time for calcul the mask position with numpy : 0.009507894515991211 nb_pixel_total : 543 time to create 1 rle with old method : 0.0009195804595947266 time for calcul the mask position with numpy : 0.009795427322387695 nb_pixel_total : 12843 time to create 1 rle with old method : 0.015636682510375977 time for calcul the mask position with numpy : 0.009426116943359375 nb_pixel_total : 15377 time to create 1 rle with old method : 0.018073558807373047 time for calcul the mask position with numpy : 0.008995771408081055 nb_pixel_total : 32359 time to create 1 rle with old method : 0.03705739974975586 create new chi : 0.5158298015594482 time to delete rle : 0.0007612705230712891 batch 1 Loaded 17 chid ids of type : 3594 +++++++++Number RLEs to save : 2903 TO DO : save crop sub photo not yet done ! save time : 0.24274063110351562 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.12766385078430176 time for calcul the mask position with numpy : 0.028960704803466797 nb_pixel_total : 1514161 time to create 1 rle with new method : 0.19399261474609375 time for calcul the mask position with numpy : 0.008633613586425781 nb_pixel_total : 12195 time to create 1 rle with old method : 0.014961957931518555 time for calcul the mask position with numpy : 0.01887059211730957 nb_pixel_total : 528828 time to create 1 rle with new method : 0.04480862617492676 time for calcul the mask position with numpy : 0.008107185363769531 nb_pixel_total : 3663 time to create 1 rle with old method : 0.004196882247924805 time for calcul the mask position with numpy : 0.008741140365600586 nb_pixel_total : 9175 time to create 1 rle with old method : 0.010726690292358398 time for calcul the mask position with numpy : 0.007841825485229492 nb_pixel_total : 5578 time to create 1 rle with old method : 0.006677865982055664 create new chi : 0.3579392433166504 time to delete rle : 0.0009806156158447266 batch 1 Loaded 11 chid ids of type : 3594 +++++++++Number RLEs to save : 4300 TO DO : save crop sub photo not yet done ! save time : 0.35731077194213867 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.21167588233947754 time for calcul the mask position with numpy : 0.09177732467651367 nb_pixel_total : 2021338 time to create 1 rle with new method : 0.4200856685638428 time for calcul the mask position with numpy : 0.007022857666015625 nb_pixel_total : 5727 time to create 1 rle with old method : 0.006619691848754883 time for calcul the mask position with numpy : 0.007001399993896484 nb_pixel_total : 5374 time to create 1 rle with old method : 0.006188392639160156 time for calcul the mask position with numpy : 0.0071909427642822266 nb_pixel_total : 9140 time to create 1 rle with old method : 0.015403270721435547 time for calcul the mask position with numpy : 0.007047891616821289 nb_pixel_total : 10113 time to create 1 rle with old method : 0.01158905029296875 time for calcul the mask position with numpy : 0.007431507110595703 nb_pixel_total : 21908 time to create 1 rle with old method : 0.026767730712890625 create new chi : 0.6274590492248535 time to delete rle : 0.000385284423828125 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 2313 TO DO : save crop sub photo not yet done ! save time : 0.19273948669433594 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.0699930191040039 time for calcul the mask position with numpy : 0.1776578426361084 nb_pixel_total : 2043452 time to create 1 rle with new method : 0.24492740631103516 time for calcul the mask position with numpy : 0.007952451705932617 nb_pixel_total : 7451 time to create 1 rle with old method : 0.01265859603881836 time for calcul the mask position with numpy : 0.008748292922973633 nb_pixel_total : 3359 time to create 1 rle with old method : 0.0060884952545166016 time for calcul the mask position with numpy : 0.00875091552734375 nb_pixel_total : 9259 time to create 1 rle with old method : 0.016342639923095703 time for calcul the mask position with numpy : 0.007928848266601562 nb_pixel_total : 10079 time to create 1 rle with old method : 0.018309831619262695 create new chi : 0.5313394069671631 time to delete rle : 0.0007913112640380859 batch 1 Loaded 9 chid ids of type : 3594 ++++++Number RLEs to save : 1988 TO DO : save crop sub photo not yet done ! save time : 0.16589856147766113 nb_obj : 5 nb_hashtags : 1 time to prepare the origin masks : 0.11222171783447266 time for calcul the mask position with numpy : 0.13148045539855957 nb_pixel_total : 1997590 time to create 1 rle with new method : 0.09842658042907715 time for calcul the mask position with numpy : 0.0065326690673828125 nb_pixel_total : 10259 time to create 1 rle with old method : 0.011790990829467773 time for calcul the mask position with numpy : 0.0068323612213134766 nb_pixel_total : 10410 time to create 1 rle with old method : 0.01188516616821289 time for calcul the mask position with numpy : 0.006704092025756836 nb_pixel_total : 10481 time to create 1 rle with old method : 0.01244211196899414 time for calcul the mask position with numpy : 0.006549835205078125 nb_pixel_total : 32341 time to create 1 rle with old method : 0.03651881217956543 time for calcul the mask position with numpy : 0.006714582443237305 nb_pixel_total : 12519 time to create 1 rle with old method : 0.016111373901367188 create new chi : 0.36374807357788086 time to delete rle : 0.0007727146148681641 batch 1 Loaded 11 chid ids of type : 3594 ++++++Number RLEs to save : 2600 TO DO : save crop sub photo not yet done ! save time : 0.3075568675994873 nb_obj : 10 nb_hashtags : 2 time to prepare the origin masks : 0.2711503505706787 time for calcul the mask position with numpy : 0.07033109664916992 nb_pixel_total : 1989592 time to create 1 rle with new method : 0.10640072822570801 time for calcul the mask position with numpy : 0.008519411087036133 nb_pixel_total : 1751 time to create 1 rle with old method : 0.0021026134490966797 time for calcul the mask position with numpy : 0.0067446231842041016 nb_pixel_total : 9204 time to create 1 rle with old method : 0.010541677474975586 time for calcul the mask position with numpy : 0.006409883499145508 nb_pixel_total : 7642 time to create 1 rle with old method : 0.008947134017944336 time for calcul the mask position with numpy : 0.007134914398193359 nb_pixel_total : 10447 time to create 1 rle with old method : 0.011909723281860352 time for calcul the mask position with numpy : 0.006552219390869141 nb_pixel_total : 3247 time to create 1 rle with old method : 0.003720521926879883 time for calcul the mask position with numpy : 0.0068204402923583984 nb_pixel_total : 1486 time to create 1 rle with old method : 0.002000570297241211 time for calcul the mask position with numpy : 0.0066454410552978516 nb_pixel_total : 18284 time to create 1 rle with old method : 0.022728681564331055 time for calcul the mask position with numpy : 0.008098840713500977 nb_pixel_total : 19646 time to create 1 rle with old method : 0.02479839324951172 time for calcul the mask position with numpy : 0.0067901611328125 nb_pixel_total : 3255 time to create 1 rle with old method : 0.003825664520263672 time for calcul the mask position with numpy : 0.006577491760253906 nb_pixel_total : 9046 time to create 1 rle with old method : 0.010636329650878906 create new chi : 0.35964202880859375 time to delete rle : 0.0005555152893066406 batch 1 Loaded 21 chid ids of type : 3594 +++++++++++++++Number RLEs to save : 3278 TO DO : save crop sub photo not yet done ! save time : 0.32562875747680664 map_output_result : {1383206553: (0.0, 'Should be the crop_list due to order', 0), 1383206500: (0.0, 'Should be the crop_list due to order', 0), 1383206445: (0.0, 'Should be the crop_list due to order', 0), 1383206398: (0.0, 'Should be the crop_list due to order', 0), 1383206057: (0.0, 'Should be the crop_list due to order', 0), 1383206003: (0.0, 'Should be the crop_list due to order', 0), 1383205955: (0.0, 'Should be the crop_list due to order', 0), 1383205908: (0.0, 'Should be the crop_list due to order', 0), 1383205860: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1383206553, 1383206500, 1383206445, 1383206398, 1383206057, 1383206003, 1383205955, 1383205908, 1383205860] Looping around the photos to save general results len do output : 9 /1383206553.Didn't retrieve data . /1383206500.Didn't retrieve data . /1383206445.Didn't retrieve data . /1383206398.Didn't retrieve data . /1383206057.Didn't retrieve data . /1383206003.Didn't retrieve data . /1383205955.Didn't retrieve data . /1383205908.Didn't retrieve data . /1383205860.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206553', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206500', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206445', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206398', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206057', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206003', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205955', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205908', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205860', None, None, None, None, None, '3694309') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.01692795753479004 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.505640983581543 time spend to save output : 0.017457008361816406 total time spend for step 3 : 9.52309799194336 step4:ventilate_hashtags_in_portfolio Thu Sep 11 10:01:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26751062 get user id for portfolio 26751062 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26751062 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','autre','pehd','background','metal','environnement','carton','mal_croppe','flou','pet_clair','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26751062 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','autre','pehd','background','metal','environnement','carton','mal_croppe','flou','pet_clair','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26751062 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','autre','pehd','background','metal','environnement','carton','mal_croppe','flou','pet_clair','pet_fonce')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26754273,26754274,26754275,26754276,26754277,26754278,26754279,26754280,26754281,26754282,26754283?tags=papier,autre,pehd,background,metal,environnement,carton,mal_croppe,flou,pet_clair,pet_fonce Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1383206553, 1383206500, 1383206445, 1383206398, 1383206057, 1383206003, 1383205955, 1383205908, 1383205860] Looping around the photos to save general results len do output : 1 /26751062. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206553', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206500', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206445', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206398', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206057', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206003', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205955', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205908', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205860', None, None, None, None, None, '3694309') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.21188020706176758 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7308926582336426 time spend to save output : 0.21220707893371582 total time spend for step 4 : 0.9430997371673584 step5:final Thu Sep 11 10:01:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1383206553: ('0.05722602666323731',), 1383206500: ('0.05722602666323731',), 1383206445: ('0.05722602666323731',), 1383206398: ('0.05722602666323731',), 1383206057: ('0.05722602666323731',), 1383206003: ('0.05722602666323731',), 1383205955: ('0.05722602666323731',), 1383205908: ('0.05722602666323731',), 1383205860: ('0.05722602666323731',)} new output for save of step final : {1383206553: ('0.05722602666323731',), 1383206500: ('0.05722602666323731',), 1383206445: ('0.05722602666323731',), 1383206398: ('0.05722602666323731',), 1383206057: ('0.05722602666323731',), 1383206003: ('0.05722602666323731',), 1383205955: ('0.05722602666323731',), 1383205908: ('0.05722602666323731',), 1383205860: ('0.05722602666323731',)} [1383206553, 1383206500, 1383206445, 1383206398, 1383206057, 1383206003, 1383205955, 1383205908, 1383205860] Looping around the photos to save general results len do output : 9 /1383206553.Didn't retrieve data . /1383206500.Didn't retrieve data . /1383206445.Didn't retrieve data . /1383206398.Didn't retrieve data . /1383206057.Didn't retrieve data . /1383206003.Didn't retrieve data . /1383205955.Didn't retrieve data . /1383205908.Didn't retrieve data . /1383205860.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206553', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206500', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206445', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206398', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206057', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206003', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205955', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205908', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205860', None, None, None, None, None, '3694309') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.019512414932250977 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.15018653869628906 time spend to save output : 0.020157337188720703 total time spend for step 5 : 0.17034387588500977 step6:blur_detection Thu Sep 11 10:01:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140.jpg resize: (1080, 1920) 1383206553 -0.26266039770835886 treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877.jpg resize: (1080, 1920) 1383206500 -0.5147463939050207 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54.jpg resize: (1080, 1920) 1383206445 2.0495535924799375 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d.jpg resize: (1080, 1920) 1383206398 -0.02054451685243059 treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681.jpg resize: (1080, 1920) 1383206057 0.4722970192162056 treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563.jpg resize: (1080, 1920) 1383206003 0.3003923329339803 treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e.jpg resize: (1080, 1920) 1383205955 2.199969044062382 treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668.jpg resize: (1080, 1920) 1383205908 0.3132386077026496 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c.jpg resize: (1080, 1920) 1383205860 -2.300155967139973 treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140_rle_crop_3955457568_0.png resize: (143, 160) 1383242504 -1.8477094028657277 treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140_rle_crop_3955457569_0.png resize: (182, 198) 1383242506 -1.2328322732309522 treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140_rle_crop_3955457570_0.png resize: (186, 264) 1383242507 -1.5652284153355382 treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457571_0.png resize: (238, 156) 1383242509 -1.973799579308356 treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457572_0.png resize: (72, 134) 1383242510 0.4510432890320023 treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457573_0.png resize: (181, 108) 1383242511 -1.1119615824195308 treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457574_0.png resize: (223, 265) 1383242514 -1.3288167162465059 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457575_0.png resize: (161, 110) 1383242515 -1.0374287227395285 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457576_0.png resize: (164, 49) 1383242517 -1.4913846172842957 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457580_0.png resize: (182, 105) 1383242518 -1.8067088495188066 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457581_0.png resize: (160, 182) 1383242519 -0.29790870742528064 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457583_0.png resize: (124, 161) 1383242520 0.31029815233667957 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457585_0.png resize: (136, 337) 1383242522 -0.9720490149325777 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457586_0.png resize: (146, 103) 1383242523 -2.0428556654153494 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457589_0.png resize: (75, 103) 1383242524 -0.8374321314427646 treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457591_0.png resize: (182, 109) 1383242526 -1.8360001340342578 treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457592_0.png resize: (45, 123) 1383242528 -1.832644625947009 treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457594_0.png resize: (143, 126) 1383242530 -0.7275745815373067 treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457596_0.png resize: (119, 131) 1383242531 -0.19629541363681977 treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457597_0.png resize: (101, 149) 1383242532 -1.0889824345161567 treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457598_0.png resize: (78, 116) 1383242534 -1.9450364323823195 treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457599_0.png resize: (96, 128) 1383242536 -0.9371865214820508 treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457600_0.png resize: (144, 114) 1383242537 -1.1533193331350375 treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457601_0.png resize: (172, 77) 1383242539 0.3149662658848787 treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457604_0.png resize: (102, 158) 1383242540 -1.4789030690424496 treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457605_0.png resize: (174, 284) 1383242541 -1.7100528814021638 treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457606_0.png resize: (134, 124) 1383242543 -1.044087748253598 treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457607_0.png resize: (105, 135) 1383242544 -0.7829641169637662 treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457608_0.png resize: (110, 123) 1383242545 -0.8485378903608436 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457609_0.png resize: (77, 126) 1383242547 4.559250659271543 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457610_0.png resize: (77, 81) 1383242548 -2.2476081818462785 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457611_0.png resize: (164, 240) 1383242549 -2.648225393502173 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457613_0.png resize: (66, 42) 1383242551 2.2261959312669393 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457614_0.png resize: (95, 49) 1383242552 -3.1710206322829264 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457615_0.png resize: (94, 178) 1383242553 -4.415473152487279 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457616_0.png resize: (98, 131) 1383242555 -1.0991349857131056 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457617_0.png resize: (121, 146) 1383242556 -2.0415050285889205 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457618_0.png resize: (40, 71) 1383242557 -1.4466034658269638 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457584_0.png resize: (114, 187) 1383242564 -0.9992610460179234 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457577_0.png resize: (156, 88) 1383242601 -1.084859379020539 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457578_0.png resize: (69, 86) 1383242602 -0.7111249687628961 treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457579_0.png resize: (152, 89) 1383242604 0.05554956486929007 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457582_0.png resize: (142, 392) 1383242605 -1.1430981036782561 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457587_0.png resize: (78, 99) 1383242607 -2.8491982550600836 treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457588_0.png resize: (169, 114) 1383242608 -2.5244656377733308 treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457590_0.png resize: (73, 109) 1383242609 0.5525156287220441 treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457593_0.png resize: (866, 857) 1383242611 -0.22832593921247216 treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457595_0.png resize: (179, 165) 1383242612 -1.0365082928870535 treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457602_0.png resize: (37, 125) 1383242613 -0.9959341102269346 treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457603_0.png resize: (80, 160) 1383242615 -2.7730928871943497 treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457612_0.png resize: (101, 232) 1383242616 -1.9210896171087934 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 60 time used for this insertion : 0.01714181900024414 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 60 time used for this insertion : 0.020638227462768555 save missing photos in datou_result : time spend for datou_step_exec : 8.110148906707764 time spend to save output : 0.04234027862548828 total time spend for step 6 : 8.152489185333252 step7:brightness Thu Sep 11 10:01:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140.jpg treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877.jpg treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54.jpg treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d.jpg treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681.jpg treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563.jpg treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e.jpg treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668.jpg treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c.jpg treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140_rle_crop_3955457568_0.png treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140_rle_crop_3955457569_0.png treat image : temp/1757577629_1251003_1383206553_52dce27eee8750ec73037007d7e8d140_rle_crop_3955457570_0.png treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457571_0.png treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457572_0.png treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457573_0.png treat image : temp/1757577629_1251003_1383206500_39f74c510f15dc0b7e03ee157dbee877_rle_crop_3955457574_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457575_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457576_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457580_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457581_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457583_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457585_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457586_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457589_0.png treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457591_0.png treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457592_0.png treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457594_0.png treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457596_0.png treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457597_0.png treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457598_0.png treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457599_0.png treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457600_0.png treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457601_0.png treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457604_0.png treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457605_0.png treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457606_0.png treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457607_0.png treat image : temp/1757577629_1251003_1383205908_db500eb2968f681914c0899ea9616668_rle_crop_3955457608_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457609_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457610_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457611_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457613_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457614_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457615_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457616_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457617_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457618_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457584_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457577_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457578_0.png treat image : temp/1757577629_1251003_1383206445_36d788081f981295cb279667139d0d54_rle_crop_3955457579_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457582_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457587_0.png treat image : temp/1757577629_1251003_1383206398_2e08bbed0e121128a7abf8549a02b39d_rle_crop_3955457588_0.png treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457590_0.png treat image : temp/1757577629_1251003_1383206057_46088a8d279d6e569155dc7ac6b6a681_rle_crop_3955457593_0.png treat image : temp/1757577629_1251003_1383206003_02120bede7d6c92691444235ed11a563_rle_crop_3955457595_0.png treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457602_0.png treat image : temp/1757577629_1251003_1383205955_f4b63ade8e7a4f86ec3ff87a5d8de97e_rle_crop_3955457603_0.png treat image : temp/1757577629_1251003_1383205860_c8d6f3fecbf906e4897b75321720c08c_rle_crop_3955457612_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 60 time used for this insertion : 0.015947341918945312 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 60 time used for this insertion : 0.12215876579284668 save missing photos in datou_result : time spend for datou_step_exec : 2.390317678451538 time spend to save output : 0.14363932609558105 total time spend for step 7 : 2.533957004547119 step8:velours_tree Thu Sep 11 10:01:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.25355100631713867 time spend to save output : 6.628036499023438e-05 total time spend for step 8 : 0.2536172866821289 step9:send_mail_cod Thu Sep 11 10:01:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26751062_11-09-2025_10_01_39.pdf 26754273 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette267542731757577699 26754274 imagette267542741757577700 26754275 imagette267542751757577700 26754276 imagette267542761757577700 26754277 imagette267542771757577700 26754279 change filename to text .imagette267542791757577700 26754280 imagette267542801757577700 26754281 imagette267542811757577700 26754282 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette267542821757577700 26754283 imagette267542831757577701 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26751062 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26754273,26754274,26754275,26754276,26754277,26754278,26754279,26754280,26754281,26754282,26754283?tags=papier,autre,pehd,background,metal,environnement,carton,mal_croppe,flou,pet_clair,pet_fonce args[1383206553] : ((1383206553, -0.26266039770835886, 492688767), (1383206553, 0.22133125677145726, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383206500] : ((1383206500, -0.5147463939050207, 492688767), (1383206500, 0.6326500040838222, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383206445] : ((1383206445, 2.0495535924799375, 492688767), (1383206445, 0.7332695272295625, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383206398] : ((1383206398, -0.02054451685243059, 492688767), (1383206398, 0.4924721249258444, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383206057] : ((1383206057, 0.4722970192162056, 492688767), (1383206057, 0.6182947689904766, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383206003] : ((1383206003, 0.3003923329339803, 492688767), (1383206003, 0.41523579833529295, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383205955] : ((1383205955, 2.199969044062382, 492688767), (1383205955, 0.7059082909886033, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383205908] : ((1383205908, 0.3132386077026496, 492688767), (1383205908, 0.5674815543655598, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com args[1383205860] : ((1383205860, -2.300155967139973, 492609224), (1383205860, 0.5096444519311903, 2107752395), '0.05722602666323731') We are sending mail with results at report@fotonower.com refus_total : 0.05722602666323731 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26751062 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751062_11-09-2025_10_01_39.pdf results_Auto_P26751062_11-09-2025_10_01_39.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751062_11-09-2025_10_01_39.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26751062','results_Auto_P26751062_11-09-2025_10_01_39.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751062_11-09-2025_10_01_39.pdf','pdf','','0.18','0.05722602666323731') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26751062

https://www.fotonower.com/image?json=false&list_photos_id=1383206553
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383206500
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383206445
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.0495535924799375)
https://www.fotonower.com/image?json=false&list_photos_id=1383206398
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383206057
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383206003
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383205955
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.199969044062382)
https://www.fotonower.com/image?json=false&list_photos_id=1383205908
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1383205860
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 5.72%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/26754273?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26754279?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26754282?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751062_11-09-2025_10_01_39.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/26754273,26754274,26754275,26754276,26754277,26754278,26754279,26754280,26754281,26754282,26754283?tags=papier,autre,pehd,background,metal,environnement,carton,mal_croppe,flou,pet_clair,pet_fonce.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 11 Sep 2025 08:01:42 GMT Content-Length: 0 Connection: close X-Message-Id: cLXLpfzjR5SB6_VR3T6voA Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1383206553, 1383206500, 1383206445, 1383206398, 1383206057, 1383206003, 1383205955, 1383205908, 1383205860] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206553', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206500', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206445', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206398', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206057', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206003', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205955', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205908', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205860', None, None, None, None, None, '3694309') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.04524493217468262 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.5907557010650635 time spend to save output : 0.0570220947265625 total time spend for step 9 : 3.647777795791626 step10:split_time_score Thu Sep 11 10:01:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 8), ('06', 1)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11092025 26751062 Nombre de photos uploadées : 9 / 23040 (0%) 11092025 26751062 Nombre de photos taguées (types de déchets): 0 / 9 (0%) 11092025 26751062 Nombre de photos taguées (volume) : 0 / 9 (0%) elapsed_time : load_data_split_time_score 3.337860107421875e-06 elapsed_time : order_list_meta_photo_and_scores 1.2874603271484375e-05 ????????? elapsed_time : fill_and_build_computed_from_old_data 0.0005774497985839844 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21032333374023438 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05722602666323731 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751062_11-09-2025_10_01_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26751062 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26751062 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26751066 order by id desc limit 1 Qualite : 0.06537318351337448 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751071_11-09-2025_09_41_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26751071 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26751071 AND mptpi.`type`=3594 To do Qualite : 0.06314996999314128 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26751073_11-09-2025_09_31_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26751073 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26751073 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11092025': {'nb_upload': 9, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1383206553, 1383206500, 1383206445, 1383206398, 1383206057, 1383206003, 1383205955, 1383205908, 1383205860] Looping around the photos to save general results len do output : 1 /26751062Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206553', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206500', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206445', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206398', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206057', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383206003', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205955', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205908', None, None, None, None, None, '3694309') ('3318', None, None, None, None, None, None, None, '3694309') ('3318', '26751062', '1383205860', None, None, None, None, None, '3694309') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.017673492431640625 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5567283630371094 time spend to save output : 0.017948150634765625 total time spend for step 10 : 1.574676513671875 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 9 set_done_treatment 35.86user 20.31system 1:19.75elapsed 70%CPU (0avgtext+0avgdata 2784124maxresident)k 504808inputs+13616outputs (739major+1502712minor)pagefaults 0swaps