python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1609465 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3735420'] with mtr_portfolio_ids : ['26944431'] and first list_photo_ids : [] new path : /proc/1609465/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 18 ; length of list_pids : 18 ; length of list_args : 18 time to download the photos : 2.825061559677124 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed Sep 17 12:40:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10589 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-17 12:40:35.789723: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-17 12:40:35.824599: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-17 12:40:35.826577: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff804000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-17 12:40:35.826613: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-17 12:40:35.831886: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-17 12:40:35.973698: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x311322d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-17 12:40:35.973766: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-17 12:40:35.975618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-17 12:40:35.976148: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-17 12:40:35.981134: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-17 12:40:36.002729: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-17 12:40:36.003438: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-17 12:40:36.039312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-17 12:40:36.044209: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-17 12:40:36.102549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-17 12:40:36.105033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-17 12:40:36.105596: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-17 12:40:36.106883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-17 12:40:36.106908: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-17 12:40:36.106924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-17 12:40:36.109574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9602 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-17 12:40:36.422651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-17 12:40:36.422763: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-17 12:40:36.422792: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-17 12:40:36.422809: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-17 12:40:36.422824: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-17 12:40:36.422839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-17 12:40:36.422853: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-17 12:40:36.422868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-17 12:40:36.424163: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-17 12:40:36.425592: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-17 12:40:36.425667: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-17 12:40:36.425683: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-17 12:40:36.425697: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-17 12:40:36.425711: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-17 12:40:36.425725: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-17 12:40:36.425739: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-17 12:40:36.425753: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-17 12:40:36.427056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-17 12:40:36.427089: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-17 12:40:36.427098: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-17 12:40:36.427106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-17 12:40:36.431130: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9602 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-17 12:40:46.129941: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-17 12:40:46.305802: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 18 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 42.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 47.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 Detection mask done ! Trying to reset tf kernel 1610228 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 416 tf kernel not reseted sub process len(results) : 18 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 18 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5705 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00020647048950195312 nb_pixel_total : 5828 time to create 1 rle with old method : 0.0067272186279296875 length of segment : 158 time for calcul the mask position with numpy : 0.002286195755004883 nb_pixel_total : 118843 time to create 1 rle with old method : 0.13877558708190918 length of segment : 595 time for calcul the mask position with numpy : 0.0004143714904785156 nb_pixel_total : 7406 time to create 1 rle with old method : 0.01006317138671875 length of segment : 190 time for calcul the mask position with numpy : 0.00019550323486328125 nb_pixel_total : 8610 time to create 1 rle with old method : 0.012149572372436523 length of segment : 131 time for calcul the mask position with numpy : 0.0002315044403076172 nb_pixel_total : 8390 time to create 1 rle with old method : 0.009983539581298828 length of segment : 171 time for calcul the mask position with numpy : 9.918212890625e-05 nb_pixel_total : 4356 time to create 1 rle with old method : 0.005049705505371094 length of segment : 62 time for calcul the mask position with numpy : 0.00019693374633789062 nb_pixel_total : 9910 time to create 1 rle with old method : 0.011605024337768555 length of segment : 163 time for calcul the mask position with numpy : 0.0003464221954345703 nb_pixel_total : 4860 time to create 1 rle with old method : 0.006247758865356445 length of segment : 167 time for calcul the mask position with numpy : 8.96453857421875e-05 nb_pixel_total : 2701 time to create 1 rle with old method : 0.0032150745391845703 length of segment : 57 time for calcul the mask position with numpy : 0.00022125244140625 nb_pixel_total : 7168 time to create 1 rle with old method : 0.008240222930908203 length of segment : 178 time for calcul the mask position with numpy : 0.00035452842712402344 nb_pixel_total : 13704 time to create 1 rle with old method : 0.01601266860961914 length of segment : 131 time for calcul the mask position with numpy : 0.00010800361633300781 nb_pixel_total : 3606 time to create 1 rle with old method : 0.004397392272949219 length of segment : 61 time for calcul the mask position with numpy : 0.000270843505859375 nb_pixel_total : 9328 time to create 1 rle with old method : 0.012977123260498047 length of segment : 160 time for calcul the mask position with numpy : 0.00016808509826660156 nb_pixel_total : 9092 time to create 1 rle with old method : 0.01060628890991211 length of segment : 151 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 3633 time to create 1 rle with old method : 0.004327058792114258 length of segment : 62 time for calcul the mask position with numpy : 0.0001659393310546875 nb_pixel_total : 7530 time to create 1 rle with old method : 0.008907318115234375 length of segment : 113 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 3107 time to create 1 rle with old method : 0.003871440887451172 length of segment : 48 time for calcul the mask position with numpy : 0.00011944770812988281 nb_pixel_total : 5246 time to create 1 rle with old method : 0.006509542465209961 length of segment : 87 time for calcul the mask position with numpy : 9.250640869140625e-05 nb_pixel_total : 4487 time to create 1 rle with old method : 0.005322933197021484 length of segment : 76 time for calcul the mask position with numpy : 0.0001232624053955078 nb_pixel_total : 5536 time to create 1 rle with old method : 0.006589174270629883 length of segment : 112 time for calcul the mask position with numpy : 0.00010848045349121094 nb_pixel_total : 4157 time to create 1 rle with old method : 0.005146980285644531 length of segment : 80 time for calcul the mask position with numpy : 0.001680612564086914 nb_pixel_total : 109969 time to create 1 rle with old method : 0.12322473526000977 length of segment : 523 time for calcul the mask position with numpy : 0.0001385211944580078 nb_pixel_total : 4255 time to create 1 rle with old method : 0.0051686763763427734 length of segment : 98 time for calcul the mask position with numpy : 0.00011324882507324219 nb_pixel_total : 4905 time to create 1 rle with old method : 0.0059049129486083984 length of segment : 65 time for calcul the mask position with numpy : 0.0002181529998779297 nb_pixel_total : 8815 time to create 1 rle with old method : 0.01047968864440918 length of segment : 185 time for calcul the mask position with numpy : 0.0001392364501953125 nb_pixel_total : 2318 time to create 1 rle with old method : 0.0043239593505859375 length of segment : 128 time for calcul the mask position with numpy : 0.00015091896057128906 nb_pixel_total : 6039 time to create 1 rle with old method : 0.00735163688659668 length of segment : 118 time for calcul the mask position with numpy : 0.00011658668518066406 nb_pixel_total : 3850 time to create 1 rle with old method : 0.004719972610473633 length of segment : 119 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 7304 time to create 1 rle with old method : 0.008717536926269531 length of segment : 89 time for calcul the mask position with numpy : 0.00011014938354492188 nb_pixel_total : 3239 time to create 1 rle with old method : 0.004019021987915039 length of segment : 82 time for calcul the mask position with numpy : 0.00023746490478515625 nb_pixel_total : 9332 time to create 1 rle with old method : 0.010805368423461914 length of segment : 172 time for calcul the mask position with numpy : 0.00010728836059570312 nb_pixel_total : 4694 time to create 1 rle with old method : 0.005734443664550781 length of segment : 73 time for calcul the mask position with numpy : 8.702278137207031e-05 nb_pixel_total : 3667 time to create 1 rle with old method : 0.004493236541748047 length of segment : 63 time for calcul the mask position with numpy : 0.00024056434631347656 nb_pixel_total : 10879 time to create 1 rle with old method : 0.012950658798217773 length of segment : 179 time for calcul the mask position with numpy : 0.0015721321105957031 nb_pixel_total : 109908 time to create 1 rle with old method : 0.14730310440063477 length of segment : 533 time for calcul the mask position with numpy : 0.00025272369384765625 nb_pixel_total : 9486 time to create 1 rle with old method : 0.011294364929199219 length of segment : 83 time for calcul the mask position with numpy : 0.00023484230041503906 nb_pixel_total : 6826 time to create 1 rle with old method : 0.008051395416259766 length of segment : 113 time for calcul the mask position with numpy : 0.00015687942504882812 nb_pixel_total : 3810 time to create 1 rle with old method : 0.004476785659790039 length of segment : 108 time for calcul the mask position with numpy : 0.0002875328063964844 nb_pixel_total : 7120 time to create 1 rle with old method : 0.008318185806274414 length of segment : 144 time for calcul the mask position with numpy : 0.0004055500030517578 nb_pixel_total : 6961 time to create 1 rle with old method : 0.008043527603149414 length of segment : 174 time for calcul the mask position with numpy : 0.00018310546875 nb_pixel_total : 5114 time to create 1 rle with old method : 0.0060961246490478516 length of segment : 79 time for calcul the mask position with numpy : 0.009493350982666016 nb_pixel_total : 383842 time to create 1 rle with new method : 0.02501964569091797 length of segment : 1030 time for calcul the mask position with numpy : 0.00027108192443847656 nb_pixel_total : 5713 time to create 1 rle with old method : 0.006279945373535156 length of segment : 128 time for calcul the mask position with numpy : 0.0004165172576904297 nb_pixel_total : 11044 time to create 1 rle with old method : 0.01226806640625 length of segment : 186 time for calcul the mask position with numpy : 0.0021393299102783203 nb_pixel_total : 83516 time to create 1 rle with old method : 0.09274148941040039 length of segment : 524 time for calcul the mask position with numpy : 0.00030040740966796875 nb_pixel_total : 5742 time to create 1 rle with old method : 0.00669097900390625 length of segment : 115 time for calcul the mask position with numpy : 0.0006592273712158203 nb_pixel_total : 19741 time to create 1 rle with old method : 0.02274346351623535 length of segment : 181 time for calcul the mask position with numpy : 0.0004086494445800781 nb_pixel_total : 12677 time to create 1 rle with old method : 0.015021085739135742 length of segment : 102 time for calcul the mask position with numpy : 0.000278472900390625 nb_pixel_total : 6539 time to create 1 rle with old method : 0.007828474044799805 length of segment : 88 time for calcul the mask position with numpy : 0.0002231597900390625 nb_pixel_total : 9273 time to create 1 rle with old method : 0.014815568923950195 length of segment : 74 time for calcul the mask position with numpy : 0.00046563148498535156 nb_pixel_total : 17173 time to create 1 rle with old method : 0.020007848739624023 length of segment : 142 time for calcul the mask position with numpy : 0.0005419254302978516 nb_pixel_total : 21737 time to create 1 rle with old method : 0.025495529174804688 length of segment : 176 time for calcul the mask position with numpy : 0.0005207061767578125 nb_pixel_total : 20901 time to create 1 rle with old method : 0.026751279830932617 length of segment : 183 time for calcul the mask position with numpy : 0.00016379356384277344 nb_pixel_total : 2242 time to create 1 rle with old method : 0.0027077198028564453 length of segment : 116 time spent for convertir_results : 2.480868101119995 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 54 chid ids of type : 3594 Number RLEs to save : 9126 save missing photos in datou_result : time spend for datou_step_exec : 30.745728015899658 time spend to save output : 0.6387956142425537 total time spend for step 1 : 31.384523630142212 step2:crop_condition Wed Sep 17 12:41:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 18 ! batch 1 Loaded 54 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 41 About to insert : list_path_to_insert length 41 new photo from crops ! About to upload 41 photos upload in portfolio : 3736932 init cache_photo without model_param we have 41 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758105665_1609465 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 41 photos in the portfolio 3736932 time of upload the photos Elapsed time : 10.68908977508545 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3736932 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1758105680_1609465 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 13 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.073121070861816 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1384138857, 1384138829, 1384138778, 1384138482, 1384138470, 1384138441, 1384138409, 1384138332, 1384137891, 1384137866, 1384137841, 1384137815, 1384137763, 1384137249, 1384137164, 1384137109, 1384137058, 1384136807] Looping around the photos to save general results len do output : 54 /1384170714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170716Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170717Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170719Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170725Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170730Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170736Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170739Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170740Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1384170813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138857', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138829', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138778', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138482', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138470', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138441', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138409', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138332', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137891', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137866', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137841', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137815', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137763', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137249', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137164', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137109', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137058', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384136807', None, None, None, None, None, '3735420') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 180 time used for this insertion : 0.13720989227294922 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.178582668304443 time spend to save output : 0.13964056968688965 total time spend for step 2 : 21.318223237991333 step3:rle_unique_nms_with_priority Wed Sep 17 12:41:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 54 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.8565464019775391 time for calcul the mask position with numpy : 0.31683802604675293 nb_pixel_total : 1941523 time to create 1 rle with new method : 0.09958434104919434 time for calcul the mask position with numpy : 0.006979465484619141 nb_pixel_total : 7406 time to create 1 rle with old method : 0.008458614349365234 time for calcul the mask position with numpy : 0.008112430572509766 nb_pixel_total : 118843 time to create 1 rle with old method : 0.1369152069091797 time for calcul the mask position with numpy : 0.011188507080078125 nb_pixel_total : 5828 time to create 1 rle with old method : 0.006683826446533203 create new chi : 0.6067502498626709 time to delete rle : 0.033925771713256836 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 2966 TO DO : save crop sub photo not yet done ! save time : 0.26000046730041504 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.10974717140197754 time for calcul the mask position with numpy : 0.35544872283935547 nb_pixel_total : 2056600 time to create 1 rle with new method : 0.16452860832214355 time for calcul the mask position with numpy : 0.006324052810668945 nb_pixel_total : 8390 time to create 1 rle with old method : 0.009241580963134766 time for calcul the mask position with numpy : 0.0061168670654296875 nb_pixel_total : 8610 time to create 1 rle with old method : 0.009530782699584961 create new chi : 0.5604865550994873 time to delete rle : 0.00025200843811035156 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1684 TO DO : save crop sub photo not yet done ! save time : 0.1418626308441162 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.13306117057800293 time for calcul the mask position with numpy : 0.11332273483276367 nb_pixel_total : 2044605 time to create 1 rle with new method : 0.2214353084564209 time for calcul the mask position with numpy : 0.009760379791259766 nb_pixel_total : 7168 time to create 1 rle with old method : 0.008157014846801758 time for calcul the mask position with numpy : 0.007920026779174805 nb_pixel_total : 2701 time to create 1 rle with old method : 0.0029816627502441406 time for calcul the mask position with numpy : 0.006095409393310547 nb_pixel_total : 4860 time to create 1 rle with old method : 0.005604267120361328 time for calcul the mask position with numpy : 0.006206989288330078 nb_pixel_total : 9910 time to create 1 rle with old method : 0.010995149612426758 time for calcul the mask position with numpy : 0.006144046783447266 nb_pixel_total : 4356 time to create 1 rle with old method : 0.004726409912109375 create new chi : 0.41461944580078125 time to delete rle : 0.0004024505615234375 batch 1 Loaded 11 chid ids of type : 3594 +++++++Number RLEs to save : 2334 TO DO : save crop sub photo not yet done ! save time : 0.1983802318572998 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.10015654563903809 time for calcul the mask position with numpy : 0.020106792449951172 nb_pixel_total : 2037870 time to create 1 rle with new method : 0.0817420482635498 time for calcul the mask position with numpy : 0.006847858428955078 nb_pixel_total : 9092 time to create 1 rle with old method : 0.014785289764404297 time for calcul the mask position with numpy : 0.0066034793853759766 nb_pixel_total : 9328 time to create 1 rle with old method : 0.015380620956420898 time for calcul the mask position with numpy : 0.006744384765625 nb_pixel_total : 3606 time to create 1 rle with old method : 0.0060443878173828125 time for calcul the mask position with numpy : 0.006795406341552734 nb_pixel_total : 13704 time to create 1 rle with old method : 0.015531301498413086 create new chi : 0.1900334358215332 time to delete rle : 0.0003306865692138672 batch 1 Loaded 9 chid ids of type : 3594 ++++++Number RLEs to save : 2086 TO DO : save crop sub photo not yet done ! save time : 0.37784671783447266 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.04831695556640625 time for calcul the mask position with numpy : 0.020416975021362305 nb_pixel_total : 2062437 time to create 1 rle with new method : 0.10503005981445312 time for calcul the mask position with numpy : 0.006410360336303711 nb_pixel_total : 7530 time to create 1 rle with old method : 0.008454322814941406 time for calcul the mask position with numpy : 0.005940437316894531 nb_pixel_total : 3633 time to create 1 rle with old method : 0.00407719612121582 create new chi : 0.1600947380065918 time to delete rle : 0.0002455711364746094 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1430 TO DO : save crop sub photo not yet done ! save time : 0.13275408744812012 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.08013057708740234 time for calcul the mask position with numpy : 0.12549710273742676 nb_pixel_total : 2060760 time to create 1 rle with new method : 0.16143107414245605 time for calcul the mask position with numpy : 0.006208181381225586 nb_pixel_total : 4487 time to create 1 rle with old method : 0.004995822906494141 time for calcul the mask position with numpy : 0.0060117244720458984 nb_pixel_total : 5246 time to create 1 rle with old method : 0.005997896194458008 time for calcul the mask position with numpy : 0.006101369857788086 nb_pixel_total : 3107 time to create 1 rle with old method : 0.0036318302154541016 create new chi : 0.33097195625305176 time to delete rle : 0.0003452301025390625 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1502 TO DO : save crop sub photo not yet done ! save time : 0.14459514617919922 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.0720815658569336 time for calcul the mask position with numpy : 0.1628124713897705 nb_pixel_total : 1953938 time to create 1 rle with new method : 0.08699655532836914 time for calcul the mask position with numpy : 0.006957054138183594 nb_pixel_total : 109969 time to create 1 rle with old method : 0.12175989151000977 time for calcul the mask position with numpy : 0.0062825679779052734 nb_pixel_total : 4157 time to create 1 rle with old method : 0.00474858283996582 time for calcul the mask position with numpy : 0.0065419673919677734 nb_pixel_total : 5536 time to create 1 rle with old method : 0.006299495697021484 create new chi : 0.413912296295166 time to delete rle : 0.0004444122314453125 batch 1 Loaded 7 chid ids of type : 3594 ++++++Number RLEs to save : 2510 TO DO : save crop sub photo not yet done ! save time : 0.2740774154663086 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.0440676212310791 time for calcul the mask position with numpy : 0.02055501937866211 nb_pixel_total : 2064440 time to create 1 rle with new method : 0.08102226257324219 time for calcul the mask position with numpy : 0.006282806396484375 nb_pixel_total : 4905 time to create 1 rle with old method : 0.0056285858154296875 time for calcul the mask position with numpy : 0.00614476203918457 nb_pixel_total : 4255 time to create 1 rle with old method : 0.004869699478149414 create new chi : 0.13202738761901855 time to delete rle : 0.00023508071899414062 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1406 TO DO : save crop sub photo not yet done ! save time : 0.13138866424560547 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.05909442901611328 time for calcul the mask position with numpy : 0.09389019012451172 nb_pixel_total : 2056428 time to create 1 rle with new method : 0.09954404830932617 time for calcul the mask position with numpy : 0.006287336349487305 nb_pixel_total : 6039 time to create 1 rle with old method : 0.006923198699951172 time for calcul the mask position with numpy : 0.0061299800872802734 nb_pixel_total : 2318 time to create 1 rle with old method : 0.002629995346069336 time for calcul the mask position with numpy : 0.0060329437255859375 nb_pixel_total : 8815 time to create 1 rle with old method : 0.009740829467773438 create new chi : 0.24326229095458984 time to delete rle : 0.0003197193145751953 batch 1 Loaded 7 chid ids of type : 3594 +++++Number RLEs to save : 1942 TO DO : save crop sub photo not yet done ! save time : 0.16400837898254395 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.04426860809326172 time for calcul the mask position with numpy : 0.2089238166809082 nb_pixel_total : 2062446 time to create 1 rle with new method : 0.08131217956542969 time for calcul the mask position with numpy : 0.006437540054321289 nb_pixel_total : 7304 time to create 1 rle with old method : 0.00826263427734375 time for calcul the mask position with numpy : 0.006411075592041016 nb_pixel_total : 3850 time to create 1 rle with old method : 0.004557371139526367 create new chi : 0.32704973220825195 time to delete rle : 0.0002777576446533203 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1496 TO DO : save crop sub photo not yet done ! save time : 0.1380617618560791 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.14283514022827148 time for calcul the mask position with numpy : 0.0669093132019043 nb_pixel_total : 2056335 time to create 1 rle with new method : 0.0953369140625 time for calcul the mask position with numpy : 0.010914087295532227 nb_pixel_total : 4694 time to create 1 rle with old method : 0.005319833755493164 time for calcul the mask position with numpy : 0.01034855842590332 nb_pixel_total : 9332 time to create 1 rle with old method : 0.01062464714050293 time for calcul the mask position with numpy : 0.006675004959106445 nb_pixel_total : 3239 time to create 1 rle with old method : 0.0037720203399658203 create new chi : 0.2216506004333496 time to delete rle : 0.0003364086151123047 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1734 TO DO : save crop sub photo not yet done ! save time : 0.20787715911865234 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 0.36150527000427246 time for calcul the mask position with numpy : 0.06518030166625977 nb_pixel_total : 1929024 time to create 1 rle with new method : 0.19674015045166016 time for calcul the mask position with numpy : 0.0064678192138671875 nb_pixel_total : 3810 time to create 1 rle with old method : 0.004360675811767578 time for calcul the mask position with numpy : 0.006600618362426758 nb_pixel_total : 6826 time to create 1 rle with old method : 0.007738351821899414 time for calcul the mask position with numpy : 0.006845712661743164 nb_pixel_total : 9486 time to create 1 rle with old method : 0.012349367141723633 time for calcul the mask position with numpy : 0.007582187652587891 nb_pixel_total : 109908 time to create 1 rle with old method : 0.13069605827331543 time for calcul the mask position with numpy : 0.006694316864013672 nb_pixel_total : 10879 time to create 1 rle with old method : 0.012644767761230469 time for calcul the mask position with numpy : 0.006579875946044922 nb_pixel_total : 3667 time to create 1 rle with old method : 0.004341840744018555 create new chi : 0.4860851764678955 time to delete rle : 0.0005929470062255859 batch 1 Loaded 13 chid ids of type : 3594 ++++++Number RLEs to save : 3238 TO DO : save crop sub photo not yet done ! save time : 0.30057358741760254 No data in photo_id : 1384137763 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.05747365951538086 time for calcul the mask position with numpy : 0.021625995635986328 nb_pixel_total : 2054405 time to create 1 rle with new method : 0.07851791381835938 time for calcul the mask position with numpy : 0.0062444210052490234 nb_pixel_total : 5114 time to create 1 rle with old method : 0.0058896541595458984 time for calcul the mask position with numpy : 0.0063304901123046875 nb_pixel_total : 6961 time to create 1 rle with old method : 0.007986068725585938 time for calcul the mask position with numpy : 0.0063343048095703125 nb_pixel_total : 7120 time to create 1 rle with old method : 0.00818634033203125 create new chi : 0.14149832725524902 time to delete rle : 0.00030732154846191406 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1874 TO DO : save crop sub photo not yet done ! save time : 0.1586458683013916 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.07486653327941895 time for calcul the mask position with numpy : 0.24062156677246094 nb_pixel_total : 1689758 time to create 1 rle with new method : 0.12355542182922363 time for calcul the mask position with numpy : 0.009181737899780273 nb_pixel_total : 383842 time to create 1 rle with new method : 0.0926506519317627 create new chi : 0.47722816467285156 time to delete rle : 0.0003120899200439453 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 3140 TO DO : save crop sub photo not yet done ! save time : 0.28191351890563965 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.3684358596801758 time for calcul the mask position with numpy : 0.07787775993347168 nb_pixel_total : 1967585 time to create 1 rle with new method : 0.11876273155212402 time for calcul the mask position with numpy : 0.0060272216796875 nb_pixel_total : 5742 time to create 1 rle with old method : 0.006310701370239258 time for calcul the mask position with numpy : 0.0066225528717041016 nb_pixel_total : 83516 time to create 1 rle with old method : 0.09224295616149902 time for calcul the mask position with numpy : 0.0063419342041015625 nb_pixel_total : 11044 time to create 1 rle with old method : 0.013885259628295898 time for calcul the mask position with numpy : 0.007110118865966797 nb_pixel_total : 5713 time to create 1 rle with old method : 0.006571054458618164 create new chi : 0.35285425186157227 time to delete rle : 0.0006709098815917969 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 2986 TO DO : save crop sub photo not yet done ! save time : 0.3102295398712158 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.06676888465881348 time for calcul the mask position with numpy : 0.06922054290771484 nb_pixel_total : 2025370 time to create 1 rle with new method : 0.2416696548461914 time for calcul the mask position with numpy : 0.006812334060668945 nb_pixel_total : 9273 time to create 1 rle with old method : 0.012678861618041992 time for calcul the mask position with numpy : 0.00632023811340332 nb_pixel_total : 6539 time to create 1 rle with old method : 0.007378578186035156 time for calcul the mask position with numpy : 0.006276369094848633 nb_pixel_total : 12677 time to create 1 rle with old method : 0.015056371688842773 time for calcul the mask position with numpy : 0.00628352165222168 nb_pixel_total : 19741 time to create 1 rle with old method : 0.022040605545043945 create new chi : 0.40444469451904297 time to delete rle : 0.0003337860107421875 batch 1 Loaded 9 chid ids of type : 3594 ++++++Number RLEs to save : 1970 TO DO : save crop sub photo not yet done ! save time : 0.22743535041809082 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.06129884719848633 time for calcul the mask position with numpy : 0.0786581039428711 nb_pixel_total : 2031198 time to create 1 rle with new method : 0.08406400680541992 time for calcul the mask position with numpy : 0.006295442581176758 nb_pixel_total : 2242 time to create 1 rle with old method : 0.0025768280029296875 time for calcul the mask position with numpy : 0.00609898567199707 nb_pixel_total : 1250 time to create 1 rle with old method : 0.0015938282012939453 time for calcul the mask position with numpy : 0.00661921501159668 nb_pixel_total : 21737 time to create 1 rle with old method : 0.024499893188476562 time for calcul the mask position with numpy : 0.0067751407623291016 nb_pixel_total : 17173 time to create 1 rle with old method : 0.01922011375427246 create new chi : 0.24753022193908691 time to delete rle : 0.0004019737243652344 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2117 TO DO : save crop sub photo not yet done ! save time : 0.18570542335510254 map_output_result : {1384138857: (0.0, 'Should be the crop_list due to order', 0), 1384138829: (0.0, 'Should be the crop_list due to order', 0), 1384138778: (0.0, 'Should be the crop_list due to order', 0), 1384138482: (0.0, 'Should be the crop_list due to order', 0), 1384138470: (0.0, 'Should be the crop_list due to order', 0), 1384138441: (0.0, 'Should be the crop_list due to order', 0), 1384138409: (0.0, 'Should be the crop_list due to order', 0), 1384138332: (0.0, 'Should be the crop_list due to order', 0), 1384137891: (0.0, 'Should be the crop_list due to order', 0), 1384137866: (0.0, 'Should be the crop_list due to order', 0), 1384137841: (0.0, 'Should be the crop_list due to order', 0), 1384137815: (0.0, 'Should be the crop_list due to order', 0), 1384137763: (0.0, 'Should be the crop_list due to order', 0.0), 1384137249: (0.0, 'Should be the crop_list due to order', 0), 1384137164: (0.0, 'Should be the crop_list due to order', 0), 1384137109: (0.0, 'Should be the crop_list due to order', 0), 1384137058: (0.0, 'Should be the crop_list due to order', 0), 1384136807: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1384138857, 1384138829, 1384138778, 1384138482, 1384138470, 1384138441, 1384138409, 1384138332, 1384137891, 1384137866, 1384137841, 1384137815, 1384137763, 1384137249, 1384137164, 1384137109, 1384137058, 1384136807] Looping around the photos to save general results len do output : 18 /1384138857.Didn't retrieve data . /1384138829.Didn't retrieve data . /1384138778.Didn't retrieve data . /1384138482.Didn't retrieve data . /1384138470.Didn't retrieve data . /1384138441.Didn't retrieve data . /1384138409.Didn't retrieve data . /1384138332.Didn't retrieve data . /1384137891.Didn't retrieve data . /1384137866.Didn't retrieve data . /1384137841.Didn't retrieve data . /1384137815.Didn't retrieve data . /1384137763.Didn't retrieve data . /1384137249.Didn't retrieve data . /1384137164.Didn't retrieve data . /1384137109.Didn't retrieve data . /1384137058.Didn't retrieve data . /1384136807.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138857', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138829', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138778', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138482', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138470', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138441', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138409', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138332', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137891', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137866', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137841', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137815', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137763', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137249', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137164', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137109', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137058', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384136807', None, None, None, None, None, '3735420') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 54 time used for this insertion : 0.015340566635131836 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.666568994522095 time spend to save output : 0.016025066375732422 total time spend for step 3 : 12.682594060897827 step4:ventilate_hashtags_in_portfolio Wed Sep 17 12:41:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26944431 get user id for portfolio 26944431 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26944431 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','mal_croppe','pet_fonce','autre','pet_clair','environnement','background','metal','carton','flou','papier')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26944431 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','mal_croppe','pet_fonce','autre','pet_clair','environnement','background','metal','carton','flou','papier')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26944431 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pehd','mal_croppe','pet_fonce','autre','pet_clair','environnement','background','metal','carton','flou','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26944794,26944795,26944796,26944797,26944798,26944799,26944800,26944801,26944802,26944803,26944804?tags=pehd,mal_croppe,pet_fonce,autre,pet_clair,environnement,background,metal,carton,flou,papier Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1384138857, 1384138829, 1384138778, 1384138482, 1384138470, 1384138441, 1384138409, 1384138332, 1384137891, 1384137866, 1384137841, 1384137815, 1384137763, 1384137249, 1384137164, 1384137109, 1384137058, 1384136807] Looping around the photos to save general results len do output : 1 /26944431. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138857', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138829', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138778', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138482', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138470', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138441', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138409', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138332', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137891', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137866', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137841', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137815', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137763', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137249', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137164', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137109', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137058', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384136807', None, None, None, None, None, '3735420') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.025858163833618164 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.7699954509735107 time spend to save output : 0.02632927894592285 total time spend for step 4 : 1.7963247299194336 step5:final Wed Sep 17 12:41:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1384138857: ('0.030984171382030176',), 1384138829: ('0.030984171382030176',), 1384138778: ('0.030984171382030176',), 1384138482: ('0.030984171382030176',), 1384138470: ('0.030984171382030176',), 1384138441: ('0.030984171382030176',), 1384138409: ('0.030984171382030176',), 1384138332: ('0.030984171382030176',), 1384137891: ('0.030984171382030176',), 1384137866: ('0.030984171382030176',), 1384137841: ('0.030984171382030176',), 1384137815: ('0.030984171382030176',), 1384137763: ('0.030984171382030176',), 1384137249: ('0.030984171382030176',), 1384137164: ('0.030984171382030176',), 1384137109: ('0.030984171382030176',), 1384137058: ('0.030984171382030176',), 1384136807: ('0.030984171382030176',)} new output for save of step final : {1384138857: ('0.030984171382030176',), 1384138829: ('0.030984171382030176',), 1384138778: ('0.030984171382030176',), 1384138482: ('0.030984171382030176',), 1384138470: ('0.030984171382030176',), 1384138441: ('0.030984171382030176',), 1384138409: ('0.030984171382030176',), 1384138332: ('0.030984171382030176',), 1384137891: ('0.030984171382030176',), 1384137866: ('0.030984171382030176',), 1384137841: ('0.030984171382030176',), 1384137815: ('0.030984171382030176',), 1384137763: ('0.030984171382030176',), 1384137249: ('0.030984171382030176',), 1384137164: ('0.030984171382030176',), 1384137109: ('0.030984171382030176',), 1384137058: ('0.030984171382030176',), 1384136807: ('0.030984171382030176',)} [1384138857, 1384138829, 1384138778, 1384138482, 1384138470, 1384138441, 1384138409, 1384138332, 1384137891, 1384137866, 1384137841, 1384137815, 1384137763, 1384137249, 1384137164, 1384137109, 1384137058, 1384136807] Looping around the photos to save general results len do output : 18 /1384138857.Didn't retrieve data . /1384138829.Didn't retrieve data . /1384138778.Didn't retrieve data . /1384138482.Didn't retrieve data . /1384138470.Didn't retrieve data . /1384138441.Didn't retrieve data . /1384138409.Didn't retrieve data . /1384138332.Didn't retrieve data . /1384137891.Didn't retrieve data . /1384137866.Didn't retrieve data . /1384137841.Didn't retrieve data . /1384137815.Didn't retrieve data . /1384137763.Didn't retrieve data . /1384137249.Didn't retrieve data . /1384137164.Didn't retrieve data . /1384137109.Didn't retrieve data . /1384137058.Didn't retrieve data . /1384136807.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138857', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138829', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138778', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138482', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138470', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138441', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138409', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138332', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137891', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137866', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137841', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137815', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137763', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137249', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137164', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137109', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137058', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384136807', None, None, None, None, None, '3735420') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 54 time used for this insertion : 0.015665769577026367 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1271512508392334 time spend to save output : 0.016744375228881836 total time spend for step 5 : 0.14389562606811523 step6:blur_detection Wed Sep 17 12:41:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50.jpg resize: (1080, 1920) 1384138857 -0.44492489144238706 treat image : temp/1758105628_1609465_1384138829_d63353fce4444864d3438959e9f4dba2.jpg resize: (1080, 1920) 1384138829 0.8580501505686615 treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec.jpg resize: (1080, 1920) 1384138778 0.1819322987558941 treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc.jpg resize: (1080, 1920) 1384138482 -4.5461699770781605 treat image : temp/1758105628_1609465_1384138470_e916bbc0f2c9e3491a31f03fcca0f170.jpg resize: (1080, 1920) 1384138470 -0.6526349732134519 treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77.jpg resize: (1080, 1920) 1384138441 -2.466907946683726 treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403.jpg resize: (1080, 1920) 1384138409 -4.266202946680125 treat image : temp/1758105628_1609465_1384138332_b52a0d464dbf593bddae52a67068985a.jpg resize: (1080, 1920) 1384138332 4.308897130711886 treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada.jpg resize: (1080, 1920) 1384137891 0.1667377970705289 treat image : temp/1758105628_1609465_1384137866_6bf7efe0da5808cdf012576cce4afcb6.jpg resize: (1080, 1920) 1384137866 -3.6859837617441613 treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027.jpg resize: (1080, 1920) 1384137841 0.3461304218935748 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b.jpg resize: (1080, 1920) 1384137815 -2.061349749455128 treat image : temp/1758105628_1609465_1384137763_61d59c89768dbd883b9e1f1d377be904.jpg resize: (1080, 1920) 1384137763 -0.7618458564961691 treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692.jpg resize: (1080, 1920) 1384137249 -0.16187665079032065 treat image : temp/1758105628_1609465_1384137164_198ac3868913b9f2c6293ed313e63eb9.jpg resize: (1080, 1920) 1384137164 0.6097811070232687 treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd.jpg resize: (1080, 1920) 1384137109 -4.453035901663794 treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11.jpg resize: (1080, 1920) 1384137058 -5.064247328536613 treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d.jpg resize: (1080, 1920) 1384136807 1.5991065419399446 treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50_rle_crop_3961246870_0.png resize: (158, 57) 1384170714 -1.7525525757693543 treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50_rle_crop_3961246872_0.png resize: (157, 106) 1384170716 -1.5531781853024644 treat image : temp/1758105628_1609465_1384138829_d63353fce4444864d3438959e9f4dba2_rle_crop_3961246874_0.png resize: (170, 110) 1384170717 -1.7471726772345857 treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246875_0.png resize: (62, 88) 1384170719 3.927660410016024 treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246877_0.png resize: (181, 186) 1384170721 -0.20657418746622622 treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246878_0.png resize: (57, 65) 1384170722 -0.9569319137777669 treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246879_0.png resize: (178, 106) 1384170724 -2.2887172418068875 treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246880_0.png resize: (115, 210) 1384170725 -2.919588196005633 treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246881_0.png resize: (60, 94) 1384170727 -3.6120764203256686 treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246882_0.png resize: (121, 167) 1384170728 -3.1246403477803497 treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246883_0.png resize: (151, 83) 1384170730 -1.2987573083841857 treat image : temp/1758105628_1609465_1384138470_e916bbc0f2c9e3491a31f03fcca0f170_rle_crop_3961246884_0.png resize: (62, 74) 1384170733 2.4117624365387327 treat image : temp/1758105628_1609465_1384138470_e916bbc0f2c9e3491a31f03fcca0f170_rle_crop_3961246885_0.png resize: (113, 147) 1384170734 -0.14040363376943776 treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77_rle_crop_3961246886_0.png resize: (45, 79) 1384170736 3.9209901206246567 treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77_rle_crop_3961246887_0.png resize: (87, 120) 1384170738 0.1772295692449995 treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403_rle_crop_3961246889_0.png resize: (112, 82) 1384170739 -1.0786398798295098 treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403_rle_crop_3961246890_0.png resize: (72, 112) 1384170740 -3.2803266855846522 treat image : temp/1758105628_1609465_1384138332_b52a0d464dbf593bddae52a67068985a_rle_crop_3961246892_0.png resize: (94, 108) 1384170741 -0.9370935960616888 treat image : temp/1758105628_1609465_1384138332_b52a0d464dbf593bddae52a67068985a_rle_crop_3961246893_0.png resize: (65, 111) 1384170742 -1.74520980288903 treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada_rle_crop_3961246894_0.png resize: (185, 108) 1384170743 -2.0474994159119966 treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada_rle_crop_3961246895_0.png resize: (69, 70) 1384170744 -2.3162030141025696 treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada_rle_crop_3961246896_0.png resize: (106, 91) 1384170745 -2.2763261122913274 treat image : temp/1758105628_1609465_1384137866_6bf7efe0da5808cdf012576cce4afcb6_rle_crop_3961246897_0.png resize: (109, 73) 1384170746 -2.240175701745826 treat image : temp/1758105628_1609465_1384137866_6bf7efe0da5808cdf012576cce4afcb6_rle_crop_3961246898_0.png resize: (89, 97) 1384170748 -2.631590781928824 treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027_rle_crop_3961246899_0.png resize: (65, 91) 1384170749 -0.6572874150067957 treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027_rle_crop_3961246900_0.png resize: (172, 104) 1384170750 -1.4797484465722244 treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027_rle_crop_3961246901_0.png resize: (71, 91) 1384170751 0.16554366277619664 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246902_0.png resize: (63, 81) 1384170752 -0.34304687725628125 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246903_0.png resize: (179, 110) 1384170753 -0.9269044844432368 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246906_0.png resize: (102, 88) 1384170754 -1.5458363656683443 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246907_0.png resize: (108, 56) 1384170755 -0.8921619715753355 treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692_rle_crop_3961246908_0.png resize: (144, 81) 1384170756 -1.0789435048530807 treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692_rle_crop_3961246909_0.png resize: (173, 100) 1384170757 -2.2121965760229836 treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692_rle_crop_3961246910_0.png resize: (79, 98) 1384170759 3.3267860959909434 treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246912_0.png resize: (127, 66) 1384170760 -3.822676991293599 treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246917_0.png resize: (102, 158) 1384170761 -1.476763872955128 treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246918_0.png resize: (80, 150) 1384170762 -4.694392734499497 treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246919_0.png resize: (62, 183) 1384170763 0.022211559585162095 treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246920_0.png resize: (142, 171) 1384170764 -0.012053471184363439 treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246921_0.png resize: (174, 296) 1384170765 -0.9368160913082582 treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246923_0.png resize: (116, 55) 1384170766 -2.8808052137181863 treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50_rle_crop_3961246871_0.png resize: (589, 346) 1384170793 -1.1138630082343466 treat image : temp/1758105628_1609465_1384138829_d63353fce4444864d3438959e9f4dba2_rle_crop_3961246873_0.png resize: (131, 96) 1384170794 -2.1426576615667527 treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246876_0.png resize: (163, 87) 1384170796 -1.4189313551528158 treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77_rle_crop_3961246888_0.png resize: (75, 76) 1384170799 -1.5340232417991824 treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403_rle_crop_3961246891_0.png resize: (516, 335) 1384170800 0.028689308795142178 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246904_0.png resize: (512, 335) 1384170801 -0.08093896943023098 treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246905_0.png resize: (59, 216) 1384170804 -2.327748511906707 treat image : temp/1758105628_1609465_1384137164_198ac3868913b9f2c6293ed313e63eb9_rle_crop_3961246911_0.png resize: (756, 810) 1384170805 0.3404861128256305 treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246913_0.png resize: (186, 90) 1384170806 -0.6998403701803294 treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246914_0.png resize: (519, 272) 1384170809 0.4983763580542978 treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246915_0.png resize: (111, 73) 1384170810 -1.3143517107471268 treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246916_0.png resize: (165, 188) 1384170812 -4.06100872834484 treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246922_0.png resize: (180, 248) 1384170813 -0.8745506861160071 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 72 time used for this insertion : 0.013950347900390625 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 72 time used for this insertion : 0.016687870025634766 save missing photos in datou_result : time spend for datou_step_exec : 14.844680309295654 time spend to save output : 0.03542661666870117 total time spend for step 6 : 14.880106925964355 step7:brightness Wed Sep 17 12:41:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50.jpg treat image : temp/1758105628_1609465_1384138829_d63353fce4444864d3438959e9f4dba2.jpg treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec.jpg treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc.jpg treat image : temp/1758105628_1609465_1384138470_e916bbc0f2c9e3491a31f03fcca0f170.jpg treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77.jpg treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403.jpg treat image : temp/1758105628_1609465_1384138332_b52a0d464dbf593bddae52a67068985a.jpg treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada.jpg treat image : temp/1758105628_1609465_1384137866_6bf7efe0da5808cdf012576cce4afcb6.jpg treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027.jpg treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b.jpg treat image : temp/1758105628_1609465_1384137763_61d59c89768dbd883b9e1f1d377be904.jpg treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692.jpg treat image : temp/1758105628_1609465_1384137164_198ac3868913b9f2c6293ed313e63eb9.jpg treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd.jpg treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11.jpg treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d.jpg treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50_rle_crop_3961246870_0.png treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50_rle_crop_3961246872_0.png treat image : temp/1758105628_1609465_1384138829_d63353fce4444864d3438959e9f4dba2_rle_crop_3961246874_0.png treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246875_0.png treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246877_0.png treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246878_0.png treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246879_0.png treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246880_0.png treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246881_0.png treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246882_0.png treat image : temp/1758105628_1609465_1384138482_697fd7f3627c7d0b04212687584c72cc_rle_crop_3961246883_0.png treat image : temp/1758105628_1609465_1384138470_e916bbc0f2c9e3491a31f03fcca0f170_rle_crop_3961246884_0.png treat image : temp/1758105628_1609465_1384138470_e916bbc0f2c9e3491a31f03fcca0f170_rle_crop_3961246885_0.png treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77_rle_crop_3961246886_0.png treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77_rle_crop_3961246887_0.png treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403_rle_crop_3961246889_0.png treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403_rle_crop_3961246890_0.png treat image : temp/1758105628_1609465_1384138332_b52a0d464dbf593bddae52a67068985a_rle_crop_3961246892_0.png treat image : temp/1758105628_1609465_1384138332_b52a0d464dbf593bddae52a67068985a_rle_crop_3961246893_0.png treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada_rle_crop_3961246894_0.png treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada_rle_crop_3961246895_0.png treat image : temp/1758105628_1609465_1384137891_59c6e8b190520a5a4984ac9293527ada_rle_crop_3961246896_0.png treat image : temp/1758105628_1609465_1384137866_6bf7efe0da5808cdf012576cce4afcb6_rle_crop_3961246897_0.png treat image : temp/1758105628_1609465_1384137866_6bf7efe0da5808cdf012576cce4afcb6_rle_crop_3961246898_0.png treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027_rle_crop_3961246899_0.png treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027_rle_crop_3961246900_0.png treat image : temp/1758105628_1609465_1384137841_19084e531422ec4ae31cf049ac5f3027_rle_crop_3961246901_0.png treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246902_0.png treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246903_0.png treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246906_0.png treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246907_0.png treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692_rle_crop_3961246908_0.png treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692_rle_crop_3961246909_0.png treat image : temp/1758105628_1609465_1384137249_8b98d48b626a499ed2411d5f3fd9a692_rle_crop_3961246910_0.png treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246912_0.png treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246917_0.png treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246918_0.png treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246919_0.png treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246920_0.png treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246921_0.png treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246923_0.png treat image : temp/1758105628_1609465_1384138857_1b2eaf8c78589e94c0d5ef298badaf50_rle_crop_3961246871_0.png treat image : temp/1758105628_1609465_1384138829_d63353fce4444864d3438959e9f4dba2_rle_crop_3961246873_0.png treat image : temp/1758105628_1609465_1384138778_2384a060310e6bc37eabe104225398ec_rle_crop_3961246876_0.png treat image : temp/1758105628_1609465_1384138441_22c7ac011cd0619c89dd47e6730cca77_rle_crop_3961246888_0.png treat image : temp/1758105628_1609465_1384138409_9cf0ea76cb5f580da2846b0bf7f22403_rle_crop_3961246891_0.png treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246904_0.png treat image : temp/1758105628_1609465_1384137815_de67f616b157125d1424cb467aac7b9b_rle_crop_3961246905_0.png treat image : temp/1758105628_1609465_1384137164_198ac3868913b9f2c6293ed313e63eb9_rle_crop_3961246911_0.png treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246913_0.png treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246914_0.png treat image : temp/1758105628_1609465_1384137109_4fee3449b0a233c1524b7c7baefee8dd_rle_crop_3961246915_0.png treat image : temp/1758105628_1609465_1384137058_6eb50cc1ca0d1fca33d1ec9e9b62fb11_rle_crop_3961246916_0.png treat image : temp/1758105628_1609465_1384136807_e0bcdc434e11206722923d2a5885536d_rle_crop_3961246922_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 72 time used for this insertion : 0.021919965744018555 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 72 time used for this insertion : 0.10614442825317383 save missing photos in datou_result : time spend for datou_step_exec : 4.200157403945923 time spend to save output : 0.13343262672424316 total time spend for step 7 : 4.333590030670166 step8:velours_tree Wed Sep 17 12:41:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.18697452545166016 time spend to save output : 3.600120544433594e-05 total time spend for step 8 : 0.1870105266571045 step9:send_mail_cod Wed Sep 17 12:41:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26944431_17-09-2025_12_41_58.pdf 26944794 imagette269447941758105718 26944795 imagette269447951758105718 26944796 imagette269447961758105718 26944797 imagette269447971758105718 26944798 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette269447981758105718 26944800 imagette269448001758105719 26944801 imagette269448011758105719 26944802 imagette269448021758105719 26944803 imagette269448031758105719 26944804 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette269448041758105719 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26944431 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26944794,26944795,26944796,26944797,26944798,26944799,26944800,26944801,26944802,26944803,26944804?tags=pehd,mal_croppe,pet_fonce,autre,pet_clair,environnement,background,metal,carton,flou,papier args[1384138857] : ((1384138857, -0.44492489144238706, 492688767), (1384138857, 0.5451155755213395, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138829] : ((1384138829, 0.8580501505686615, 492688767), (1384138829, 0.7012411066597717, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138778] : ((1384138778, 0.1819322987558941, 492688767), (1384138778, 0.5758456246179773, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138482] : ((1384138482, -4.5461699770781605, 492609224), (1384138482, 0.38472569033158405, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138470] : ((1384138470, -0.6526349732134519, 492688767), (1384138470, 0.5091110163940786, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138441] : ((1384138441, -2.466907946683726, 492609224), (1384138441, 0.4124366709650621, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138409] : ((1384138409, -4.266202946680125, 492609224), (1384138409, 0.4184214113011819, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384138332] : ((1384138332, 4.308897130711886, 492688767), (1384138332, 0.3822467181541705, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137891] : ((1384137891, 0.1667377970705289, 492688767), (1384137891, 0.40119786737587776, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137866] : ((1384137866, -3.6859837617441613, 492609224), (1384137866, 0.22671615872937623, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137841] : ((1384137841, 0.3461304218935748, 492688767), (1384137841, 0.6344213469176994, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137815] : ((1384137815, -2.061349749455128, 492609224), (1384137815, 0.4173902191455902, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137763] : ((1384137763, -0.7618458564961691, 492688767), (1384137763, 0.5227862592301342, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137249] : ((1384137249, -0.16187665079032065, 492688767), (1384137249, 0.5457932102483914, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137164] : ((1384137164, 0.6097811070232687, 492688767), (1384137164, 0.7637616422991647, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137109] : ((1384137109, -4.453035901663794, 492609224), (1384137109, 0.674922229235205, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384137058] : ((1384137058, -5.064247328536613, 492609224), (1384137058, 0.3624021822493937, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com args[1384136807] : ((1384136807, 1.5991065419399446, 492688767), (1384136807, 0.4385774383672339, 2107752395), '0.030984171382030176') We are sending mail with results at report@fotonower.com refus_total : 0.030984171382030176 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26944431 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26944431_17-09-2025_12_41_58.pdf results_Auto_P26944431_17-09-2025_12_41_58.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26944431_17-09-2025_12_41_58.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26944431','results_Auto_P26944431_17-09-2025_12_41_58.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26944431_17-09-2025_12_41_58.pdf','pdf','','0.27','0.030984171382030176') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26944431

https://www.fotonower.com/image?json=false&list_photos_id=1384138857
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138829
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138778
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138482
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138470
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138441
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138409
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384138332
La photo est trop floue, merci de reprendre une photo.(avec le score = 4.308897130711886)
https://www.fotonower.com/image?json=false&list_photos_id=1384137891
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137866
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137841
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137815
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137763
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137249
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137164
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137109
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384137058
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1384136807
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.5991065419399446)

Dans ces conditions,le taux de refus est: 3.10%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/26944798?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26944804?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26944431_17-09-2025_12_41_58.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/26944794,26944795,26944796,26944797,26944798,26944799,26944800,26944801,26944802,26944803,26944804?tags=pehd,mal_croppe,pet_fonce,autre,pet_clair,environnement,background,metal,carton,flou,papier.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 17 Sep 2025 10:42:02 GMT Content-Length: 0 Connection: close X-Message-Id: -ger0JxyTF-7f03jd1yQSQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1384138857, 1384138829, 1384138778, 1384138482, 1384138470, 1384138441, 1384138409, 1384138332, 1384137891, 1384137866, 1384137841, 1384137815, 1384137763, 1384137249, 1384137164, 1384137109, 1384137058, 1384136807] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138857', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138829', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138778', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138482', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138470', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138441', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138409', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138332', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137891', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137866', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137841', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137815', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137763', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137249', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137164', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137109', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137058', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384136807', None, None, None, None, None, '3735420') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.09198856353759766 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.842425584793091 time spend to save output : 0.09226846694946289 total time spend for step 9 : 3.9346940517425537 step10:split_time_score Wed Sep 17 12:42:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 18),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 17092025 26944431 Nombre de photos uploadées : 18 / 23040 (0%) 17092025 26944431 Nombre de photos taguées (types de déchets): 0 / 18 (0%) 17092025 26944431 Nombre de photos taguées (volume) : 0 / 18 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ?????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0008997917175292969 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21974492073059082 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.10858108874133637 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26931981_17-09-2025_09_30_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26931981 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26931981 AND mptpi.`type`=3594 To do Qualite : 0.07002329282407407 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26931985_17-09-2025_08_51_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26931985 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26931985 AND mptpi.`type`=3594 To do Qualite : 0.060776120580808085 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26931986_17-09-2025_08_41_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26931986 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26931986 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26944425 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26941005 order by id desc limit 1 Qualite : 0.12312721032664604 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26941009_17-09-2025_11_41_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26941009 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26941009 AND mptpi.`type`=3594 To do Qualite : 0.030984171382030176 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26944431_17-09-2025_12_41_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26944431 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26944431 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'17092025': {'nb_upload': 18, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1384138857, 1384138829, 1384138778, 1384138482, 1384138470, 1384138441, 1384138409, 1384138332, 1384137891, 1384137866, 1384137841, 1384137815, 1384137763, 1384137249, 1384137164, 1384137109, 1384137058, 1384136807] Looping around the photos to save general results len do output : 1 /26944431Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138857', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138829', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138778', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138482', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138470', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138441', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138409', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384138332', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137891', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137866', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137841', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137815', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137763', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137249', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137164', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137109', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384137058', None, None, None, None, None, '3735420') ('3318', None, None, None, None, None, None, None, '3735420') ('3318', '26944431', '1384136807', None, None, None, None, None, '3735420') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.020035505294799805 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.6524760723114014 time spend to save output : 0.020335912704467773 total time spend for step 10 : 2.672811985015869 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 18 set_done_treatment 49.05user 23.40system 1:39.85elapsed 72%CPU (0avgtext+0avgdata 3050388maxresident)k 1949672inputs+19576outputs (7394major+2113950minor)pagefaults 0swaps