python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1759117 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3556071'] with mtr_portfolio_ids : ['26075827'] and first list_photo_ids : [] new path : /proc/1759117/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 17 ; length of list_pids : 17 ; length of list_args : 17 time to download the photos : 2.144477605819702 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed Aug 20 12:10:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10617 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-20 12:10:34.849851: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-20 12:10:34.876482: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-20 12:10:34.878642: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f9368000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-20 12:10:34.878702: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-20 12:10:34.882828: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-20 12:10:35.066486: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1444b4a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-20 12:10:35.066550: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-20 12:10:35.068052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-20 12:10:35.068509: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 12:10:35.071830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 12:10:35.074928: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-20 12:10:35.075457: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-20 12:10:35.078409: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-20 12:10:35.079477: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-20 12:10:35.084173: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-20 12:10:35.085651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-20 12:10:35.085730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 12:10:35.086486: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-20 12:10:35.086501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-20 12:10:35.086509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-20 12:10:35.087811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9835 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-20 12:10:35.340177: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-20 12:10:35.340313: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 12:10:35.340802: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 12:10:35.340822: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-20 12:10:35.340836: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-20 12:10:35.340851: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-20 12:10:35.340864: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-20 12:10:35.340879: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-20 12:10:35.342172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-20 12:10:35.343375: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-20 12:10:35.343408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-20 12:10:35.343426: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 12:10:35.343441: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-20 12:10:35.343457: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-20 12:10:35.343472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-20 12:10:35.343488: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-20 12:10:35.343503: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-20 12:10:35.344675: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-20 12:10:35.344709: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-20 12:10:35.344720: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-20 12:10:35.344729: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-20 12:10:35.345949: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9835 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-20 12:10:43.005430: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-20 12:10:43.198735: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 17 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 19.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 59.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 50.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 1759711 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4291 tf kernel not reseted sub process len(results) : 17 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 17 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2350 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.00017499923706054688 nb_pixel_total : 3915 time to create 1 rle with old method : 0.0048961639404296875 length of segment : 65 time for calcul the mask position with numpy : 0.00012159347534179688 nb_pixel_total : 5668 time to create 1 rle with old method : 0.006755828857421875 length of segment : 65 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 1746 time to create 1 rle with old method : 0.002169370651245117 length of segment : 49 time for calcul the mask position with numpy : 0.012752294540405273 nb_pixel_total : 726490 time to create 1 rle with new method : 0.09116840362548828 length of segment : 989 time for calcul the mask position with numpy : 0.0015616416931152344 nb_pixel_total : 105940 time to create 1 rle with old method : 0.12087416648864746 length of segment : 531 time for calcul the mask position with numpy : 0.0015137195587158203 nb_pixel_total : 92626 time to create 1 rle with old method : 0.10484504699707031 length of segment : 499 time for calcul the mask position with numpy : 8.702278137207031e-05 nb_pixel_total : 1731 time to create 1 rle with old method : 0.0022156238555908203 length of segment : 38 time for calcul the mask position with numpy : 0.0018224716186523438 nb_pixel_total : 109028 time to create 1 rle with old method : 0.1255626678466797 length of segment : 470 time for calcul the mask position with numpy : 0.00010633468627929688 nb_pixel_total : 4247 time to create 1 rle with old method : 0.005197763442993164 length of segment : 77 time for calcul the mask position with numpy : 6.961822509765625e-05 nb_pixel_total : 1777 time to create 1 rle with old method : 0.002283811569213867 length of segment : 40 time for calcul the mask position with numpy : 0.0008692741394042969 nb_pixel_total : 56126 time to create 1 rle with old method : 0.06567907333374023 length of segment : 320 time for calcul the mask position with numpy : 9.34600830078125e-05 nb_pixel_total : 3118 time to create 1 rle with old method : 0.0038383007049560547 length of segment : 42 time for calcul the mask position with numpy : 0.0007112026214599609 nb_pixel_total : 55445 time to create 1 rle with old method : 0.06337952613830566 length of segment : 309 time for calcul the mask position with numpy : 0.00034999847412109375 nb_pixel_total : 18254 time to create 1 rle with old method : 0.021246671676635742 length of segment : 166 time for calcul the mask position with numpy : 0.010277748107910156 nb_pixel_total : 733394 time to create 1 rle with new method : 0.0212247371673584 length of segment : 993 time for calcul the mask position with numpy : 0.0007557868957519531 nb_pixel_total : 40057 time to create 1 rle with old method : 0.04530811309814453 length of segment : 230 time for calcul the mask position with numpy : 0.0016431808471679688 nb_pixel_total : 112259 time to create 1 rle with old method : 0.13419890403747559 length of segment : 538 time for calcul the mask position with numpy : 0.0004527568817138672 nb_pixel_total : 23517 time to create 1 rle with old method : 0.027712345123291016 length of segment : 167 time for calcul the mask position with numpy : 0.0003294944763183594 nb_pixel_total : 12419 time to create 1 rle with old method : 0.01460409164428711 length of segment : 210 time for calcul the mask position with numpy : 0.0002620220184326172 nb_pixel_total : 14530 time to create 1 rle with old method : 0.0176088809967041 length of segment : 106 time for calcul the mask position with numpy : 0.0032138824462890625 nb_pixel_total : 110555 time to create 1 rle with old method : 0.1289520263671875 length of segment : 550 time for calcul the mask position with numpy : 0.0003535747528076172 nb_pixel_total : 8760 time to create 1 rle with old method : 0.009798526763916016 length of segment : 95 time for calcul the mask position with numpy : 0.0004057884216308594 nb_pixel_total : 11735 time to create 1 rle with old method : 0.013511896133422852 length of segment : 109 time for calcul the mask position with numpy : 0.00014519691467285156 nb_pixel_total : 1837 time to create 1 rle with old method : 0.002233743667602539 length of segment : 35 time for calcul the mask position with numpy : 0.00024056434631347656 nb_pixel_total : 8200 time to create 1 rle with old method : 0.009405136108398438 length of segment : 168 time for calcul the mask position with numpy : 9.775161743164062e-05 nb_pixel_total : 1401 time to create 1 rle with old method : 0.0016644001007080078 length of segment : 37 time for calcul the mask position with numpy : 0.00032901763916015625 nb_pixel_total : 4400 time to create 1 rle with old method : 0.005126237869262695 length of segment : 101 time for calcul the mask position with numpy : 0.0035276412963867188 nb_pixel_total : 122370 time to create 1 rle with old method : 0.1331033706665039 length of segment : 539 time for calcul the mask position with numpy : 0.0007383823394775391 nb_pixel_total : 16819 time to create 1 rle with old method : 0.01904749870300293 length of segment : 138 time for calcul the mask position with numpy : 0.0026581287384033203 nb_pixel_total : 107122 time to create 1 rle with old method : 0.11638545989990234 length of segment : 518 time for calcul the mask position with numpy : 0.0001690387725830078 nb_pixel_total : 2483 time to create 1 rle with old method : 0.0028564929962158203 length of segment : 48 time for calcul the mask position with numpy : 0.00030350685119628906 nb_pixel_total : 7220 time to create 1 rle with old method : 0.008980274200439453 length of segment : 95 time for calcul the mask position with numpy : 0.0003421306610107422 nb_pixel_total : 4505 time to create 1 rle with old method : 0.005452394485473633 length of segment : 104 time for calcul the mask position with numpy : 0.00022482872009277344 nb_pixel_total : 4009 time to create 1 rle with old method : 0.004821062088012695 length of segment : 65 time for calcul the mask position with numpy : 0.0006792545318603516 nb_pixel_total : 10371 time to create 1 rle with old method : 0.01231074333190918 length of segment : 180 time for calcul the mask position with numpy : 0.0008389949798583984 nb_pixel_total : 36160 time to create 1 rle with old method : 0.04855227470397949 length of segment : 329 time for calcul the mask position with numpy : 0.00021600723266601562 nb_pixel_total : 3856 time to create 1 rle with old method : 0.004889011383056641 length of segment : 52 time for calcul the mask position with numpy : 0.0005002021789550781 nb_pixel_total : 19520 time to create 1 rle with old method : 0.023409366607666016 length of segment : 189 time for calcul the mask position with numpy : 0.00020003318786621094 nb_pixel_total : 4380 time to create 1 rle with old method : 0.005369663238525391 length of segment : 104 time for calcul the mask position with numpy : 0.00047016143798828125 nb_pixel_total : 18474 time to create 1 rle with old method : 0.021944046020507812 length of segment : 169 time for calcul the mask position with numpy : 0.00031685829162597656 nb_pixel_total : 13784 time to create 1 rle with old method : 0.016617298126220703 length of segment : 127 time for calcul the mask position with numpy : 0.00015616416931152344 nb_pixel_total : 4219 time to create 1 rle with old method : 0.005325794219970703 length of segment : 70 time for calcul the mask position with numpy : 0.0002892017364501953 nb_pixel_total : 8611 time to create 1 rle with old method : 0.010124921798706055 length of segment : 169 time for calcul the mask position with numpy : 0.0005195140838623047 nb_pixel_total : 9784 time to create 1 rle with old method : 0.01160430908203125 length of segment : 176 time for calcul the mask position with numpy : 0.0008563995361328125 nb_pixel_total : 22408 time to create 1 rle with old method : 0.026522397994995117 length of segment : 195 time for calcul the mask position with numpy : 0.0005815029144287109 nb_pixel_total : 10518 time to create 1 rle with old method : 0.012093544006347656 length of segment : 181 time for calcul the mask position with numpy : 0.00020813941955566406 nb_pixel_total : 4903 time to create 1 rle with old method : 0.0061168670654296875 length of segment : 59 time for calcul the mask position with numpy : 0.0003352165222167969 nb_pixel_total : 4330 time to create 1 rle with old method : 0.005196571350097656 length of segment : 107 time for calcul the mask position with numpy : 0.002853870391845703 nb_pixel_total : 112268 time to create 1 rle with old method : 0.1264338493347168 length of segment : 517 time for calcul the mask position with numpy : 0.00040650367736816406 nb_pixel_total : 8577 time to create 1 rle with old method : 0.010024785995483398 length of segment : 101 time for calcul the mask position with numpy : 0.00018978118896484375 nb_pixel_total : 4676 time to create 1 rle with old method : 0.00542759895324707 length of segment : 128 time for calcul the mask position with numpy : 0.001291036605834961 nb_pixel_total : 44595 time to create 1 rle with old method : 0.05042386054992676 length of segment : 344 time for calcul the mask position with numpy : 0.0005626678466796875 nb_pixel_total : 25817 time to create 1 rle with old method : 0.029559612274169922 length of segment : 116 time for calcul the mask position with numpy : 0.0003261566162109375 nb_pixel_total : 4148 time to create 1 rle with old method : 0.004908323287963867 length of segment : 125 time for calcul the mask position with numpy : 0.0006968975067138672 nb_pixel_total : 20476 time to create 1 rle with old method : 0.024103403091430664 length of segment : 213 time for calcul the mask position with numpy : 0.0005743503570556641 nb_pixel_total : 24908 time to create 1 rle with old method : 0.028901338577270508 length of segment : 131 time for calcul the mask position with numpy : 0.0004146099090576172 nb_pixel_total : 9965 time to create 1 rle with old method : 0.011728286743164062 length of segment : 117 time for calcul the mask position with numpy : 0.0011181831359863281 nb_pixel_total : 37641 time to create 1 rle with old method : 0.042491912841796875 length of segment : 270 time for calcul the mask position with numpy : 0.001062154769897461 nb_pixel_total : 32624 time to create 1 rle with old method : 0.03630995750427246 length of segment : 314 time for calcul the mask position with numpy : 0.00017905235290527344 nb_pixel_total : 5306 time to create 1 rle with old method : 0.006445169448852539 length of segment : 67 time for calcul the mask position with numpy : 0.001489400863647461 nb_pixel_total : 53014 time to create 1 rle with old method : 0.05953812599182129 length of segment : 332 time for calcul the mask position with numpy : 0.001287221908569336 nb_pixel_total : 54202 time to create 1 rle with old method : 0.07534980773925781 length of segment : 253 time for calcul the mask position with numpy : 0.0003001689910888672 nb_pixel_total : 5101 time to create 1 rle with old method : 0.008873224258422852 length of segment : 51 time for calcul the mask position with numpy : 0.000659942626953125 nb_pixel_total : 18907 time to create 1 rle with old method : 0.021863460540771484 length of segment : 121 time for calcul the mask position with numpy : 0.00025773048400878906 nb_pixel_total : 4203 time to create 1 rle with old method : 0.005112171173095703 length of segment : 98 time for calcul the mask position with numpy : 0.000640869140625 nb_pixel_total : 17188 time to create 1 rle with old method : 0.020583152770996094 length of segment : 156 time for calcul the mask position with numpy : 0.0004341602325439453 nb_pixel_total : 9581 time to create 1 rle with old method : 0.011255025863647461 length of segment : 188 time for calcul the mask position with numpy : 0.000705718994140625 nb_pixel_total : 22085 time to create 1 rle with old method : 0.0290224552154541 length of segment : 158 time for calcul the mask position with numpy : 0.00023412704467773438 nb_pixel_total : 4704 time to create 1 rle with old method : 0.005315303802490234 length of segment : 108 time for calcul the mask position with numpy : 0.0005114078521728516 nb_pixel_total : 16325 time to create 1 rle with old method : 0.01876378059387207 length of segment : 157 time for calcul the mask position with numpy : 0.0014328956604003906 nb_pixel_total : 58509 time to create 1 rle with old method : 0.06538844108581543 length of segment : 246 time for calcul the mask position with numpy : 0.0007236003875732422 nb_pixel_total : 24002 time to create 1 rle with old method : 0.036879539489746094 length of segment : 147 time for calcul the mask position with numpy : 0.002473115921020508 nb_pixel_total : 93172 time to create 1 rle with old method : 0.10596275329589844 length of segment : 534 time for calcul the mask position with numpy : 0.0003662109375 nb_pixel_total : 10603 time to create 1 rle with old method : 0.012237071990966797 length of segment : 139 time for calcul the mask position with numpy : 0.0007381439208984375 nb_pixel_total : 17602 time to create 1 rle with old method : 0.021259069442749023 length of segment : 189 time for calcul the mask position with numpy : 0.0002830028533935547 nb_pixel_total : 9950 time to create 1 rle with old method : 0.013000726699829102 length of segment : 123 time for calcul the mask position with numpy : 0.0003581047058105469 nb_pixel_total : 3106 time to create 1 rle with old method : 0.0038614273071289062 length of segment : 127 time spent for convertir_results : 4.74813175201416 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 77 chid ids of type : 3594 Number RLEs to save : 16183 save missing photos in datou_result : time spend for datou_step_exec : 30.698770999908447 time spend to save output : 1.2641468048095703 total time spend for step 1 : 31.962917804718018 step2:crop_condition Wed Aug 20 12:11:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 17 ! batch 1 Loaded 77 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 41 About to insert : list_path_to_insert length 41 new photo from crops ! About to upload 41 photos upload in portfolio : 3736932 init cache_photo without model_param we have 41 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755684667_1759117 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 41 photos in the portfolio 3736932 time of upload the photos Elapsed time : 12.433646202087402 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3736932 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755684680_1759117 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 5 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.8022851943969727 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755684682_1759117 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5194973945617676 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 27 About to insert : list_path_to_insert length 27 new photo from crops ! About to upload 27 photos upload in portfolio : 3736932 init cache_photo without model_param we have 27 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755684695_1759117 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 27 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.014150381088257 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755684702_1759117 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5956292152404785 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1755684703_1759117 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.8986470699310303 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1378008790, 1378008788, 1378008767, 1378008759, 1378008752, 1378008733, 1378008732, 1378008731, 1378008595, 1378008570, 1378008515, 1378008358, 1378008335, 1378008295, 1378008205, 1378008172, 1378008036] Looping around the photos to save general results len do output : 77 /1378023350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1378023479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008790', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008788', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008767', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008759', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008752', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008733', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008732', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008731', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008595', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008570', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008515', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008358', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008335', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008295', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008205', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008172', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008036', None, None, None, None, None, '3556071') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 248 time used for this insertion : 0.04337906837463379 save_final save missing photos in datou_result : time spend for datou_step_exec : 40.33859586715698 time spend to save output : 0.04678964614868164 total time spend for step 2 : 40.385385513305664 step3:rle_unique_nms_with_priority Wed Aug 20 12:11:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 77 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.6130378246307373 time for calcul the mask position with numpy : 0.040012359619140625 nb_pixel_total : 1229841 time to create 1 rle with new method : 0.30952978134155273 time for calcul the mask position with numpy : 0.007596731185913086 nb_pixel_total : 105940 time to create 1 rle with old method : 0.14405035972595215 time for calcul the mask position with numpy : 0.016432523727416992 nb_pixel_total : 726490 time to create 1 rle with new method : 0.07853913307189941 time for calcul the mask position with numpy : 0.006659030914306641 nb_pixel_total : 1746 time to create 1 rle with old method : 0.001990079879760742 time for calcul the mask position with numpy : 0.006409168243408203 nb_pixel_total : 5668 time to create 1 rle with old method : 0.006433010101318359 time for calcul the mask position with numpy : 0.006737470626831055 nb_pixel_total : 3915 time to create 1 rle with old method : 0.004438638687133789 create new chi : 0.6465287208557129 time to delete rle : 0.01792597770690918 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 4478 TO DO : save crop sub photo not yet done ! save time : 0.29763317108154297 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.0711984634399414 time for calcul the mask position with numpy : 0.022235631942749023 nb_pixel_total : 1979243 time to create 1 rle with new method : 0.1731264591217041 time for calcul the mask position with numpy : 0.005872964859008789 nb_pixel_total : 1731 time to create 1 rle with old method : 0.001895904541015625 time for calcul the mask position with numpy : 0.006819725036621094 nb_pixel_total : 92626 time to create 1 rle with old method : 0.10048842430114746 create new chi : 0.3107919692993164 time to delete rle : 0.00032019615173339844 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2154 TO DO : save crop sub photo not yet done ! save time : 0.1897270679473877 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.08140325546264648 time for calcul the mask position with numpy : 0.5046181678771973 nb_pixel_total : 1899321 time to create 1 rle with new method : 0.08163690567016602 time for calcul the mask position with numpy : 0.007166147232055664 nb_pixel_total : 3101 time to create 1 rle with old method : 0.003428220748901367 time for calcul the mask position with numpy : 0.01099705696105957 nb_pixel_total : 56126 time to create 1 rle with old method : 0.06178426742553711 time for calcul the mask position with numpy : 0.010375499725341797 nb_pixel_total : 1777 time to create 1 rle with old method : 0.0019795894622802734 time for calcul the mask position with numpy : 0.010063648223876953 nb_pixel_total : 4247 time to create 1 rle with old method : 0.004781007766723633 time for calcul the mask position with numpy : 0.006972074508666992 nb_pixel_total : 109028 time to create 1 rle with old method : 0.11819720268249512 create new chi : 0.8324284553527832 time to delete rle : 0.00041031837463378906 batch 1 Loaded 11 chid ids of type : 3594 +++++++Number RLEs to save : 2960 TO DO : save crop sub photo not yet done ! save time : 0.2785613536834717 nb_obj : 5 nb_hashtags : 1 time to prepare the origin masks : 0.1157386302947998 time for calcul the mask position with numpy : 0.014735937118530273 nb_pixel_total : 1114191 time to create 1 rle with new method : 0.09370255470275879 time for calcul the mask position with numpy : 0.007692098617553711 nb_pixel_total : 112259 time to create 1 rle with old method : 0.12256956100463867 time for calcul the mask position with numpy : 0.006662845611572266 nb_pixel_total : 40057 time to create 1 rle with old method : 0.0441439151763916 time for calcul the mask position with numpy : 0.013131380081176758 nb_pixel_total : 733394 time to create 1 rle with new method : 0.2658545970916748 time for calcul the mask position with numpy : 0.006870269775390625 nb_pixel_total : 18254 time to create 1 rle with old method : 0.020319223403930664 time for calcul the mask position with numpy : 0.006295680999755859 nb_pixel_total : 55445 time to create 1 rle with old method : 0.060027360916137695 create new chi : 0.6772074699401855 time to delete rle : 0.0008335113525390625 batch 1 Loaded 11 chid ids of type : 3594 ++++++++++Number RLEs to save : 5552 TO DO : save crop sub photo not yet done ! save time : 0.3757352828979492 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.06187081336975098 time for calcul the mask position with numpy : 0.060311079025268555 nb_pixel_total : 1912579 time to create 1 rle with new method : 0.1419227123260498 time for calcul the mask position with numpy : 0.006983518600463867 nb_pixel_total : 110555 time to create 1 rle with old method : 0.12122845649719238 time for calcul the mask position with numpy : 0.0065174102783203125 nb_pixel_total : 14530 time to create 1 rle with old method : 0.01817607879638672 time for calcul the mask position with numpy : 0.0061490535736083984 nb_pixel_total : 12419 time to create 1 rle with old method : 0.013492345809936523 time for calcul the mask position with numpy : 0.009663105010986328 nb_pixel_total : 23517 time to create 1 rle with old method : 0.025524377822875977 create new chi : 0.42215394973754883 time to delete rle : 0.0009815692901611328 batch 1 Loaded 9 chid ids of type : 3594 ++++++Number RLEs to save : 3146 TO DO : save crop sub photo not yet done ! save time : 0.26935648918151855 nb_obj : 8 nb_hashtags : 3 time to prepare the origin masks : 0.5456149578094482 time for calcul the mask position with numpy : 0.17884540557861328 nb_pixel_total : 1898078 time to create 1 rle with new method : 0.2736058235168457 time for calcul the mask position with numpy : 0.006374835968017578 nb_pixel_total : 16819 time to create 1 rle with old method : 0.018705129623413086 time for calcul the mask position with numpy : 0.008320808410644531 nb_pixel_total : 122370 time to create 1 rle with old method : 0.1338186264038086 time for calcul the mask position with numpy : 0.006131649017333984 nb_pixel_total : 4400 time to create 1 rle with old method : 0.004946470260620117 time for calcul the mask position with numpy : 0.006295680999755859 nb_pixel_total : 1401 time to create 1 rle with old method : 0.0015840530395507812 time for calcul the mask position with numpy : 0.006330013275146484 nb_pixel_total : 8200 time to create 1 rle with old method : 0.009161233901977539 time for calcul the mask position with numpy : 0.006190776824951172 nb_pixel_total : 1837 time to create 1 rle with old method : 0.002095937728881836 time for calcul the mask position with numpy : 0.006147146224975586 nb_pixel_total : 11735 time to create 1 rle with old method : 0.013176202774047852 time for calcul the mask position with numpy : 0.0063724517822265625 nb_pixel_total : 8760 time to create 1 rle with old method : 0.00973367691040039 create new chi : 0.7078399658203125 time to delete rle : 0.0005884170532226562 batch 1 Loaded 17 chid ids of type : 3594 ++++++++Number RLEs to save : 3524 TO DO : save crop sub photo not yet done ! save time : 0.27494239807128906 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03429007530212402 time for calcul the mask position with numpy : 0.0183870792388916 nb_pixel_total : 1966478 time to create 1 rle with new method : 0.02758336067199707 time for calcul the mask position with numpy : 0.006402254104614258 nb_pixel_total : 107122 time to create 1 rle with old method : 0.12196969985961914 create new chi : 0.17467927932739258 time to delete rle : 0.0004894733428955078 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2116 TO DO : save crop sub photo not yet done ! save time : 0.20107531547546387 nb_obj : 5 nb_hashtags : 1 time to prepare the origin masks : 0.34699201583862305 time for calcul the mask position with numpy : 0.05922818183898926 nb_pixel_total : 2045012 time to create 1 rle with new method : 0.13037991523742676 time for calcul the mask position with numpy : 0.006117105484008789 nb_pixel_total : 10371 time to create 1 rle with old method : 0.01125478744506836 time for calcul the mask position with numpy : 0.006035327911376953 nb_pixel_total : 4009 time to create 1 rle with old method : 0.004452943801879883 time for calcul the mask position with numpy : 0.0056819915771484375 nb_pixel_total : 4505 time to create 1 rle with old method : 0.004951000213623047 time for calcul the mask position with numpy : 0.005944490432739258 nb_pixel_total : 7220 time to create 1 rle with old method : 0.008149862289428711 time for calcul the mask position with numpy : 0.005991697311401367 nb_pixel_total : 2483 time to create 1 rle with old method : 0.0027289390563964844 create new chi : 0.26215124130249023 time to delete rle : 0.0004379749298095703 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 2064 TO DO : save crop sub photo not yet done ! save time : 0.206254243850708 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.04488372802734375 time for calcul the mask position with numpy : 0.019960641860961914 nb_pixel_total : 2033584 time to create 1 rle with new method : 0.14298558235168457 time for calcul the mask position with numpy : 0.006153583526611328 nb_pixel_total : 3856 time to create 1 rle with old method : 0.004231452941894531 time for calcul the mask position with numpy : 0.006066322326660156 nb_pixel_total : 36160 time to create 1 rle with old method : 0.0390782356262207 create new chi : 0.22553586959838867 time to delete rle : 0.00027370452880859375 batch 1 Loaded 5 chid ids of type : 3594 ++++++++Number RLEs to save : 1842 TO DO : save crop sub photo not yet done ! save time : 0.15162181854248047 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 0.12162184715270996 time for calcul the mask position with numpy : 0.12115955352783203 nb_pixel_total : 2004612 time to create 1 rle with new method : 0.14251399040222168 time for calcul the mask position with numpy : 0.005933284759521484 nb_pixel_total : 8611 time to create 1 rle with old method : 0.009154319763183594 time for calcul the mask position with numpy : 0.005728721618652344 nb_pixel_total : 4219 time to create 1 rle with old method : 0.004643678665161133 time for calcul the mask position with numpy : 0.005727529525756836 nb_pixel_total : 13784 time to create 1 rle with old method : 0.01481175422668457 time for calcul the mask position with numpy : 0.005892753601074219 nb_pixel_total : 18474 time to create 1 rle with old method : 0.02001333236694336 time for calcul the mask position with numpy : 0.0067217350006103516 nb_pixel_total : 4380 time to create 1 rle with old method : 0.004735469818115234 time for calcul the mask position with numpy : 0.0058667659759521484 nb_pixel_total : 19520 time to create 1 rle with old method : 0.02079033851623535 create new chi : 0.3847043514251709 time to delete rle : 0.0003199577331542969 batch 1 Loaded 13 chid ids of type : 3594 +++++++++Number RLEs to save : 2736 TO DO : save crop sub photo not yet done ! save time : 0.25185537338256836 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 0.2479867935180664 time for calcul the mask position with numpy : 0.05187487602233887 nb_pixel_total : 1851541 time to create 1 rle with new method : 0.31454014778137207 time for calcul the mask position with numpy : 0.0063745975494384766 nb_pixel_total : 44595 time to create 1 rle with old method : 0.046413421630859375 time for calcul the mask position with numpy : 0.0058443546295166016 nb_pixel_total : 4676 time to create 1 rle with old method : 0.005198001861572266 time for calcul the mask position with numpy : 0.005803823471069336 nb_pixel_total : 8577 time to create 1 rle with old method : 0.009189844131469727 time for calcul the mask position with numpy : 0.006376981735229492 nb_pixel_total : 112268 time to create 1 rle with old method : 0.11644339561462402 time for calcul the mask position with numpy : 0.005997180938720703 nb_pixel_total : 4330 time to create 1 rle with old method : 0.004609584808349609 time for calcul the mask position with numpy : 0.0057141780853271484 nb_pixel_total : 4903 time to create 1 rle with old method : 0.005162954330444336 time for calcul the mask position with numpy : 0.0057964324951171875 nb_pixel_total : 10518 time to create 1 rle with old method : 0.014142036437988281 time for calcul the mask position with numpy : 0.006275177001953125 nb_pixel_total : 22408 time to create 1 rle with old method : 0.023816347122192383 time for calcul the mask position with numpy : 0.0059549808502197266 nb_pixel_total : 9784 time to create 1 rle with old method : 0.01077723503112793 create new chi : 0.6661665439605713 time to delete rle : 0.0008280277252197266 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++++Number RLEs to save : 4696 TO DO : save crop sub photo not yet done ! save time : 0.3331589698791504 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.0693361759185791 time for calcul the mask position with numpy : 0.019547462463378906 nb_pixel_total : 2043635 time to create 1 rle with new method : 0.03977370262145996 time for calcul the mask position with numpy : 0.006341457366943359 nb_pixel_total : 4148 time to create 1 rle with old method : 0.004765033721923828 time for calcul the mask position with numpy : 0.006448030471801758 nb_pixel_total : 25817 time to create 1 rle with old method : 0.028773069381713867 create new chi : 0.1059255599975586 time to delete rle : 0.00027441978454589844 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1562 TO DO : save crop sub photo not yet done ! save time : 0.3954169750213623 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.038709163665771484 time for calcul the mask position with numpy : 0.020412206649780273 nb_pixel_total : 2028216 time to create 1 rle with new method : 0.03425121307373047 time for calcul the mask position with numpy : 0.005980253219604492 nb_pixel_total : 24908 time to create 1 rle with old method : 0.02610301971435547 time for calcul the mask position with numpy : 0.005880594253540039 nb_pixel_total : 20476 time to create 1 rle with old method : 0.021794557571411133 create new chi : 0.11470794677734375 time to delete rle : 0.0003044605255126953 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 1768 TO DO : save crop sub photo not yet done ! save time : 0.1396925449371338 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.06806755065917969 time for calcul the mask position with numpy : 0.018451452255249023 nb_pixel_total : 1988064 time to create 1 rle with new method : 0.2784850597381592 time for calcul the mask position with numpy : 0.005802631378173828 nb_pixel_total : 5306 time to create 1 rle with old method : 0.005682706832885742 time for calcul the mask position with numpy : 0.0058019161224365234 nb_pixel_total : 32624 time to create 1 rle with old method : 0.034404754638671875 time for calcul the mask position with numpy : 0.005887031555175781 nb_pixel_total : 37641 time to create 1 rle with old method : 0.04051017761230469 time for calcul the mask position with numpy : 0.006069183349609375 nb_pixel_total : 9965 time to create 1 rle with old method : 0.010703325271606445 create new chi : 0.41883039474487305 time to delete rle : 0.000431060791015625 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2616 TO DO : save crop sub photo not yet done ! save time : 0.2973952293395996 nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 0.6260786056518555 time for calcul the mask position with numpy : 0.055716514587402344 nb_pixel_total : 1892380 time to create 1 rle with new method : 0.10377168655395508 time for calcul the mask position with numpy : 0.006579399108886719 nb_pixel_total : 22085 time to create 1 rle with old method : 0.03373146057128906 time for calcul the mask position with numpy : 0.0067136287689208984 nb_pixel_total : 9581 time to create 1 rle with old method : 0.0165555477142334 time for calcul the mask position with numpy : 0.006905794143676758 nb_pixel_total : 17188 time to create 1 rle with old method : 0.025474071502685547 time for calcul the mask position with numpy : 0.006525993347167969 nb_pixel_total : 4203 time to create 1 rle with old method : 0.0047473907470703125 time for calcul the mask position with numpy : 0.00659942626953125 nb_pixel_total : 15846 time to create 1 rle with old method : 0.017740488052368164 time for calcul the mask position with numpy : 0.006519794464111328 nb_pixel_total : 5101 time to create 1 rle with old method : 0.005700588226318359 time for calcul the mask position with numpy : 0.006751537322998047 nb_pixel_total : 54202 time to create 1 rle with old method : 0.060498952865600586 time for calcul the mask position with numpy : 0.006729602813720703 nb_pixel_total : 53014 time to create 1 rle with old method : 0.0590062141418457 create new chi : 0.44770050048828125 time to delete rle : 0.0007138252258300781 batch 1 Loaded 17 chid ids of type : 3594 +++++++++++Number RLEs to save : 3760 TO DO : save crop sub photo not yet done ! save time : 0.27376604080200195 nb_obj : 8 nb_hashtags : 5 time to prepare the origin masks : 0.4422445297241211 time for calcul the mask position with numpy : 0.04728078842163086 nb_pixel_total : 1838733 time to create 1 rle with new method : 0.21608448028564453 time for calcul the mask position with numpy : 0.006014585494995117 nb_pixel_total : 9950 time to create 1 rle with old method : 0.010515213012695312 time for calcul the mask position with numpy : 0.0057485103607177734 nb_pixel_total : 17602 time to create 1 rle with old method : 0.018822193145751953 time for calcul the mask position with numpy : 0.005961179733276367 nb_pixel_total : 10603 time to create 1 rle with old method : 0.013659000396728516 time for calcul the mask position with numpy : 0.007480621337890625 nb_pixel_total : 93172 time to create 1 rle with old method : 0.1399860382080078 time for calcul the mask position with numpy : 0.006576061248779297 nb_pixel_total : 24002 time to create 1 rle with old method : 0.02768707275390625 time for calcul the mask position with numpy : 0.007047176361083984 nb_pixel_total : 58509 time to create 1 rle with old method : 0.06493186950683594 time for calcul the mask position with numpy : 0.00676417350769043 nb_pixel_total : 16325 time to create 1 rle with old method : 0.018200159072875977 time for calcul the mask position with numpy : 0.0068433284759521484 nb_pixel_total : 4704 time to create 1 rle with old method : 0.0052945613861083984 create new chi : 0.6247084140777588 time to delete rle : 0.0007588863372802734 batch 1 Loaded 17 chid ids of type : 3594 ++++++++++Number RLEs to save : 4366 TO DO : save crop sub photo not yet done ! save time : 0.44446778297424316 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03237199783325195 time for calcul the mask position with numpy : 0.019037961959838867 nb_pixel_total : 2070494 time to create 1 rle with new method : 0.02745223045349121 time for calcul the mask position with numpy : 0.005912303924560547 nb_pixel_total : 3106 time to create 1 rle with old method : 0.0034418106079101562 create new chi : 0.05605936050415039 time to delete rle : 0.00021719932556152344 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1334 TO DO : save crop sub photo not yet done ! save time : 0.10973119735717773 map_output_result : {1378008790: (0.0, 'Should be the crop_list due to order', 0), 1378008788: (0.0, 'Should be the crop_list due to order', 0), 1378008767: (0.0, 'Should be the crop_list due to order', 0), 1378008759: (0.0, 'Should be the crop_list due to order', 0), 1378008752: (0.0, 'Should be the crop_list due to order', 0), 1378008733: (0.0, 'Should be the crop_list due to order', 0), 1378008732: (0.0, 'Should be the crop_list due to order', 0), 1378008731: (0.0, 'Should be the crop_list due to order', 0), 1378008595: (0.0, 'Should be the crop_list due to order', 0), 1378008570: (0.0, 'Should be the crop_list due to order', 0), 1378008515: (0.0, 'Should be the crop_list due to order', 0), 1378008358: (0.0, 'Should be the crop_list due to order', 0), 1378008335: (0.0, 'Should be the crop_list due to order', 0), 1378008295: (0.0, 'Should be the crop_list due to order', 0), 1378008205: (0.0, 'Should be the crop_list due to order', 0), 1378008172: (0.0, 'Should be the crop_list due to order', 0), 1378008036: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1378008790, 1378008788, 1378008767, 1378008759, 1378008752, 1378008733, 1378008732, 1378008731, 1378008595, 1378008570, 1378008515, 1378008358, 1378008335, 1378008295, 1378008205, 1378008172, 1378008036] Looping around the photos to save general results len do output : 17 /1378008790.Didn't retrieve data . /1378008788.Didn't retrieve data . /1378008767.Didn't retrieve data . /1378008759.Didn't retrieve data . /1378008752.Didn't retrieve data . /1378008733.Didn't retrieve data . /1378008732.Didn't retrieve data . /1378008731.Didn't retrieve data . /1378008595.Didn't retrieve data . /1378008570.Didn't retrieve data . /1378008515.Didn't retrieve data . /1378008358.Didn't retrieve data . /1378008335.Didn't retrieve data . /1378008295.Didn't retrieve data . /1378008205.Didn't retrieve data . /1378008172.Didn't retrieve data . /1378008036.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008790', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008788', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008767', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008759', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008752', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008733', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008732', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008731', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008595', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008570', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008515', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008358', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008335', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008295', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008205', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008172', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008036', None, None, None, None, None, '3556071') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 51 time used for this insertion : 0.017394304275512695 save_final save missing photos in datou_result : time spend for datou_step_exec : 15.727524518966675 time spend to save output : 0.01796245574951172 total time spend for step 3 : 15.745486974716187 step4:ventilate_hashtags_in_portfolio Wed Aug 20 12:12:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26075827 get user id for portfolio 26075827 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26075827 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_fonce','pehd','background','environnement','flou','metal','pet_clair','mal_croppe','autre','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26075827 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_fonce','pehd','background','environnement','flou','metal','pet_clair','mal_croppe','autre','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26075827 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_fonce','pehd','background','environnement','flou','metal','pet_clair','mal_croppe','autre','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/26075944,26075945,26075946,26075947,26075948,26075949,26075950,26075951,26075952,26075953,26075954?tags=papier,pet_fonce,pehd,background,environnement,flou,metal,pet_clair,mal_croppe,autre,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1378008790, 1378008788, 1378008767, 1378008759, 1378008752, 1378008733, 1378008732, 1378008731, 1378008595, 1378008570, 1378008515, 1378008358, 1378008335, 1378008295, 1378008205, 1378008172, 1378008036] Looping around the photos to save general results len do output : 1 /26075827. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008790', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008788', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008767', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008759', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008752', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008733', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008732', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008731', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008595', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008570', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008515', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008358', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008335', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008295', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008205', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008172', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008036', None, None, None, None, None, '3556071') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.05517315864562988 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6882214546203613 time spend to save output : 0.05564284324645996 total time spend for step 4 : 1.7438642978668213 step5:final Wed Aug 20 12:12:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1378008790: ('0.09801646468772696',), 1378008788: ('0.09801646468772696',), 1378008767: ('0.09801646468772696',), 1378008759: ('0.09801646468772696',), 1378008752: ('0.09801646468772696',), 1378008733: ('0.09801646468772696',), 1378008732: ('0.09801646468772696',), 1378008731: ('0.09801646468772696',), 1378008595: ('0.09801646468772696',), 1378008570: ('0.09801646468772696',), 1378008515: ('0.09801646468772696',), 1378008358: ('0.09801646468772696',), 1378008335: ('0.09801646468772696',), 1378008295: ('0.09801646468772696',), 1378008205: ('0.09801646468772696',), 1378008172: ('0.09801646468772696',), 1378008036: ('0.09801646468772696',)} new output for save of step final : {1378008790: ('0.09801646468772696',), 1378008788: ('0.09801646468772696',), 1378008767: ('0.09801646468772696',), 1378008759: ('0.09801646468772696',), 1378008752: ('0.09801646468772696',), 1378008733: ('0.09801646468772696',), 1378008732: ('0.09801646468772696',), 1378008731: ('0.09801646468772696',), 1378008595: ('0.09801646468772696',), 1378008570: ('0.09801646468772696',), 1378008515: ('0.09801646468772696',), 1378008358: ('0.09801646468772696',), 1378008335: ('0.09801646468772696',), 1378008295: ('0.09801646468772696',), 1378008205: ('0.09801646468772696',), 1378008172: ('0.09801646468772696',), 1378008036: ('0.09801646468772696',)} [1378008790, 1378008788, 1378008767, 1378008759, 1378008752, 1378008733, 1378008732, 1378008731, 1378008595, 1378008570, 1378008515, 1378008358, 1378008335, 1378008295, 1378008205, 1378008172, 1378008036] Looping around the photos to save general results len do output : 17 /1378008790.Didn't retrieve data . /1378008788.Didn't retrieve data . /1378008767.Didn't retrieve data . /1378008759.Didn't retrieve data . /1378008752.Didn't retrieve data . /1378008733.Didn't retrieve data . /1378008732.Didn't retrieve data . /1378008731.Didn't retrieve data . /1378008595.Didn't retrieve data . /1378008570.Didn't retrieve data . /1378008515.Didn't retrieve data . /1378008358.Didn't retrieve data . /1378008335.Didn't retrieve data . /1378008295.Didn't retrieve data . /1378008205.Didn't retrieve data . /1378008172.Didn't retrieve data . /1378008036.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008790', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008788', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008767', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008759', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008752', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008733', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008732', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008731', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008595', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008570', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008515', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008358', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008335', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008295', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008205', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008172', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008036', None, None, None, None, None, '3556071') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 51 time used for this insertion : 0.02347540855407715 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12081480026245117 time spend to save output : 0.0242769718170166 total time spend for step 5 : 0.14509177207946777 step6:blur_detection Wed Aug 20 12:12:02 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4.jpg resize: (1080, 1920) 1378008790 1.1421976184179525 treat image : temp/1755684630_1759117_1378008788_7b9e0ce3167fc76dafbc3da108db16a1.jpg resize: (1080, 1920) 1378008788 -0.6075425510225507 treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc.jpg resize: (1080, 1920) 1378008767 -3.2303396982863153 treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8.jpg resize: (1080, 1920) 1378008759 -1.4454122245302994 treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c.jpg resize: (1080, 1920) 1378008752 -0.8195509487313748 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece.jpg resize: (1080, 1920) 1378008733 -0.5969251627440688 treat image : temp/1755684630_1759117_1378008732_9b8339968287578650acefc3b85b5847.jpg resize: (1080, 1920) 1378008732 -0.9938047240144782 treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935.jpg resize: (1080, 1920) 1378008731 -0.9625334989297292 treat image : temp/1755684630_1759117_1378008595_5b5b5af7c984c2aacbf43accaff8b11f.jpg resize: (1080, 1920) 1378008595 -1.4811292323886174 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9.jpg resize: (1080, 1920) 1378008570 -3.210455229956183 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba.jpg resize: (1080, 1920) 1378008515 0.09787461520746246 treat image : temp/1755684630_1759117_1378008358_a0dcbad1f90782578649e6bec2c55b6f.jpg resize: (1080, 1920) 1378008358 -2.176914757745654 treat image : temp/1755684630_1759117_1378008335_1d5c23fe7b070e2540631a3d1451782f.jpg resize: (1080, 1920) 1378008335 1.4457476629545258 treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8.jpg resize: (1080, 1920) 1378008295 -3.7716676713989634 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2.jpg resize: (1080, 1920) 1378008205 -3.455626636982021 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535.jpg resize: (1080, 1920) 1378008172 -3.9738477003262793 treat image : temp/1755684630_1759117_1378008036_06776fe48e9c18199818a71c775ef3b4.jpg resize: (1080, 1920) 1378008036 -0.6035043407521379 treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533453_0.png resize: (64, 84) 1378023350 -0.7037806861175501 treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533454_0.png resize: (64, 101) 1378023352 9.872507508703004 treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533462_0.png resize: (40, 51) 1378023353 -1.907828092346631 treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533464_0.png resize: (39, 87) 1378023354 -3.0639635179237605 treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533470_0.png resize: (157, 255) 1378023356 -0.5990798598311904 treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533471_0.png resize: (210, 147) 1378023357 -2.464530630354521 treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533472_0.png resize: (100, 218) 1378023358 0.8496376040912316 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533474_0.png resize: (95, 124) 1378023360 -2.0018807977538433 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533476_0.png resize: (34, 75) 1378023361 0.34372495496931005 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533477_0.png resize: (168, 78) 1378023362 -1.1950206043129386 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533478_0.png resize: (37, 45) 1378023364 20.0 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533479_0.png resize: (101, 84) 1378023365 -2.09720311952173 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533481_0.png resize: (125, 172) 1378023366 -0.7958710166583123 treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533483_0.png resize: (48, 68) 1378023368 0.8047893888526697 treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533484_0.png resize: (95, 110) 1378023369 -1.6076332605470334 treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533485_0.png resize: (104, 84) 1378023370 -2.1273372402014403 treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533486_0.png resize: (65, 91) 1378023372 -0.3768937167773927 treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533487_0.png resize: (179, 104) 1378023373 -0.7026098412600752 treat image : temp/1755684630_1759117_1378008595_5b5b5af7c984c2aacbf43accaff8b11f_rle_crop_3921533488_0.png resize: (232, 290) 1378023374 -1.6235105387870676 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533490_0.png resize: (188, 175) 1378023376 -0.8516390026637886 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533491_0.png resize: (104, 83) 1378023377 -2.2637739304305606 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533493_0.png resize: (126, 138) 1378023378 -2.911905673298203 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533495_0.png resize: (169, 108) 1378023380 -1.7282444485132253 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533496_0.png resize: (176, 92) 1378023381 -1.8693418539232844 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533498_0.png resize: (181, 102) 1378023382 -0.4216844536565861 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533499_0.png resize: (59, 103) 1378023384 1.051875332044658 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533500_0.png resize: (107, 80) 1378023385 -2.1874206534049043 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533502_0.png resize: (101, 138) 1378023386 -1.9136324671312024 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533503_0.png resize: (128, 55) 1378023388 -1.3074484446590582 treat image : temp/1755684630_1759117_1378008358_a0dcbad1f90782578649e6bec2c55b6f_rle_crop_3921533506_0.png resize: (123, 78) 1378023389 -2.0542020694165535 treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533509_0.png resize: (115, 131) 1378023390 -1.1870927680399646 treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533510_0.png resize: (270, 256) 1378023393 -2.982812341617794 treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533511_0.png resize: (313, 191) 1378023394 -2.019026801020177 treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533512_0.png resize: (67, 122) 1378023395 -1.5449915678880282 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533513_0.png resize: (329, 256) 1378023397 -0.8047029739074956 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533517_0.png resize: (98, 80) 1378023398 -1.9416339830867049 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533518_0.png resize: (146, 245) 1378023399 -1.2601981945565388 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533519_0.png resize: (147, 106) 1378023401 -2.0849922005103325 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533521_0.png resize: (108, 84) 1378023402 -2.2863377731903225 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533527_0.png resize: (150, 256) 1378023403 -1.0861964623115912 treat image : temp/1755684630_1759117_1378008036_06776fe48e9c18199818a71c775ef3b4_rle_crop_3921533529_0.png resize: (119, 76) 1378023405 -2.7595078209317667 treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533460_0.png resize: (470, 370) 1378023422 -0.3687986594920405 treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533461_0.png resize: (77, 71) 1378023423 -1.8896586861133775 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533475_0.png resize: (109, 155) 1378023425 -1.6331188661981386 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533497_0.png resize: (169, 200) 1378023426 -1.652202249866472 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533522_0.png resize: (152, 151) 1378023427 -0.9668262634011349 treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533455_0.png resize: (49, 52) 1378023432 -2.8596964266100113 treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533456_0.png resize: (983, 1030) 1378023447 0.17874100975067514 treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533457_0.png resize: (520, 343) 1378023448 0.16514402194209735 treat image : temp/1755684630_1759117_1378008788_7b9e0ce3167fc76dafbc3da108db16a1_rle_crop_3921533458_0.png resize: (485, 338) 1378023450 -0.24631766945262257 treat image : temp/1755684630_1759117_1378008788_7b9e0ce3167fc76dafbc3da108db16a1_rle_crop_3921533459_0.png resize: (38, 60) 1378023451 -0.1661091477871716 treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533463_0.png resize: (196, 470) 1378023452 -2.934483233754257 treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533465_0.png resize: (272, 280) 1378023454 -0.9522969344550659 treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533466_0.png resize: (147, 214) 1378023455 -0.8926233497519399 treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533467_0.png resize: (973, 1001) 1378023456 0.5664409462825664 treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533468_0.png resize: (224, 249) 1378023457 -1.3224340161609318 treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533469_0.png resize: (530, 338) 1378023458 0.22766317516479329 treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533473_0.png resize: (523, 346) 1378023459 -0.495040945003673 treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533480_0.png resize: (530, 377) 1378023460 0.3359583439624188 treat image : temp/1755684630_1759117_1378008732_9b8339968287578650acefc3b85b5847_rle_crop_3921533482_0.png resize: (515, 341) 1378023461 0.025852588654807416 treat image : temp/1755684630_1759117_1378008595_5b5b5af7c984c2aacbf43accaff8b11f_rle_crop_3921533489_0.png resize: (52, 101) 1378023462 -0.6712568607372472 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533492_0.png resize: (169, 149) 1378023464 -1.9463983339405289 treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533494_0.png resize: (56, 101) 1378023465 -3.535336936376357 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533501_0.png resize: (515, 336) 1378023466 0.2913683840627847 treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533504_0.png resize: (325, 172) 1378023467 -0.912953400301887 treat image : temp/1755684630_1759117_1378008358_a0dcbad1f90782578649e6bec2c55b6f_rle_crop_3921533505_0.png resize: (113, 312) 1378023468 -3.176121764753223 treat image : temp/1755684630_1759117_1378008335_1d5c23fe7b070e2540631a3d1451782f_rle_crop_3921533507_0.png resize: (197, 239) 1378023469 -0.3918550463639688 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533514_0.png resize: (249, 304) 1378023470 -3.298392530177337 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533515_0.png resize: (51, 122) 1378023471 -2.3648448696525715 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533516_0.png resize: (121, 209) 1378023472 -1.8125497082678106 treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533520_0.png resize: (158, 192) 1378023473 -2.1846484312333163 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533523_0.png resize: (245, 314) 1378023474 -4.660411708630984 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533524_0.png resize: (138, 214) 1378023475 -1.646405732460835 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533525_0.png resize: (525, 294) 1378023476 0.1904082750682787 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533528_0.png resize: (123, 122) 1378023477 -3.8809093642693506 treat image : temp/1755684630_1759117_1378008335_1d5c23fe7b070e2540631a3d1451782f_rle_crop_3921533508_0.png resize: (130, 248) 1378023478 0.2800719721893719 treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533526_0.png resize: (139, 113) 1378023479 -2.363889166371609 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 94 time used for this insertion : 0.016196489334106445 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 94 time used for this insertion : 0.017461299896240234 save missing photos in datou_result : time spend for datou_step_exec : 15.090136766433716 time spend to save output : 0.03917717933654785 total time spend for step 6 : 15.129313945770264 step7:brightness Wed Aug 20 12:12:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4.jpg treat image : temp/1755684630_1759117_1378008788_7b9e0ce3167fc76dafbc3da108db16a1.jpg treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc.jpg treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8.jpg treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c.jpg treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece.jpg treat image : temp/1755684630_1759117_1378008732_9b8339968287578650acefc3b85b5847.jpg treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935.jpg treat image : temp/1755684630_1759117_1378008595_5b5b5af7c984c2aacbf43accaff8b11f.jpg treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9.jpg treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba.jpg treat image : temp/1755684630_1759117_1378008358_a0dcbad1f90782578649e6bec2c55b6f.jpg treat image : temp/1755684630_1759117_1378008335_1d5c23fe7b070e2540631a3d1451782f.jpg treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8.jpg treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2.jpg treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535.jpg treat image : temp/1755684630_1759117_1378008036_06776fe48e9c18199818a71c775ef3b4.jpg treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533453_0.png treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533454_0.png treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533462_0.png treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533464_0.png treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533470_0.png treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533471_0.png treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533472_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533474_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533476_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533477_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533478_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533479_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533481_0.png treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533483_0.png treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533484_0.png treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533485_0.png treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533486_0.png treat image : temp/1755684630_1759117_1378008731_95e670f7d7d3d2c74be81baa8ccd9935_rle_crop_3921533487_0.png treat image : temp/1755684630_1759117_1378008595_5b5b5af7c984c2aacbf43accaff8b11f_rle_crop_3921533488_0.png treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533490_0.png treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533491_0.png treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533493_0.png treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533495_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533496_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533498_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533499_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533500_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533502_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533503_0.png treat image : temp/1755684630_1759117_1378008358_a0dcbad1f90782578649e6bec2c55b6f_rle_crop_3921533506_0.png treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533509_0.png treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533510_0.png treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533511_0.png treat image : temp/1755684630_1759117_1378008295_d80aa245a669a43d6d08a4021c860ca8_rle_crop_3921533512_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533513_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533517_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533518_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533519_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533521_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533527_0.png treat image : temp/1755684630_1759117_1378008036_06776fe48e9c18199818a71c775ef3b4_rle_crop_3921533529_0.png treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533460_0.png treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533461_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533475_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533497_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533522_0.png treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533455_0.png treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533456_0.png treat image : temp/1755684630_1759117_1378008790_2dc3e2383b5ae62c87b1b9e6cf6139f4_rle_crop_3921533457_0.png treat image : temp/1755684630_1759117_1378008788_7b9e0ce3167fc76dafbc3da108db16a1_rle_crop_3921533458_0.png treat image : temp/1755684630_1759117_1378008788_7b9e0ce3167fc76dafbc3da108db16a1_rle_crop_3921533459_0.png treat image : temp/1755684630_1759117_1378008767_9b0d549dfe15280ae9ab78b4e761b8cc_rle_crop_3921533463_0.png treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533465_0.png treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533466_0.png treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533467_0.png treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533468_0.png treat image : temp/1755684630_1759117_1378008759_ab435d3f94d1b1b4e18e33012cf9cca8_rle_crop_3921533469_0.png treat image : temp/1755684630_1759117_1378008752_f49061b8b8baf26c7f616ddc6308f33c_rle_crop_3921533473_0.png treat image : temp/1755684630_1759117_1378008733_d28f974c42c7db01832ea475e52aeece_rle_crop_3921533480_0.png treat image : temp/1755684630_1759117_1378008732_9b8339968287578650acefc3b85b5847_rle_crop_3921533482_0.png treat image : temp/1755684630_1759117_1378008595_5b5b5af7c984c2aacbf43accaff8b11f_rle_crop_3921533489_0.png treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533492_0.png treat image : temp/1755684630_1759117_1378008570_57c99a8a8599d35aa73245706cf2c1f9_rle_crop_3921533494_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533501_0.png treat image : temp/1755684630_1759117_1378008515_75ee434233cf463cc604b2cee982f0ba_rle_crop_3921533504_0.png treat image : temp/1755684630_1759117_1378008358_a0dcbad1f90782578649e6bec2c55b6f_rle_crop_3921533505_0.png treat image : temp/1755684630_1759117_1378008335_1d5c23fe7b070e2540631a3d1451782f_rle_crop_3921533507_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533514_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533515_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533516_0.png treat image : temp/1755684630_1759117_1378008205_e95015fa22b2480a392a4fc593a532a2_rle_crop_3921533520_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533523_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533524_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533525_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533528_0.png treat image : temp/1755684630_1759117_1378008335_1d5c23fe7b070e2540631a3d1451782f_rle_crop_3921533508_0.png treat image : temp/1755684630_1759117_1378008172_ea7211b11d1faa4501747de8b873e535_rle_crop_3921533526_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 94 time used for this insertion : 0.02338862419128418 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 94 time used for this insertion : 0.23772954940795898 save missing photos in datou_result : time spend for datou_step_exec : 4.394590139389038 time spend to save output : 0.2668757438659668 total time spend for step 7 : 4.661465883255005 step8:velours_tree Wed Aug 20 12:12:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.10696792602539062 time spend to save output : 5.125999450683594e-05 total time spend for step 8 : 0.10701918601989746 step9:send_mail_cod Wed Aug 20 12:12:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26075827_20-08-2025_12_12_22.pdf 26075944 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette260759441755684742 26075945 change filename to text .change filename to text .imagette260759451755684743 26075946 imagette260759461755684743 26075947 imagette260759471755684743 26075949 imagette260759491755684743 26075950 change filename to text .imagette260759501755684743 26075951 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette260759511755684743 26075952 imagette260759521755684744 26075953 change filename to text .imagette260759531755684744 26075954 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette260759541755684744 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26075827 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26075944,26075945,26075946,26075947,26075948,26075949,26075950,26075951,26075952,26075953,26075954?tags=papier,pet_fonce,pehd,background,environnement,flou,metal,pet_clair,mal_croppe,autre,carton args[1378008790] : ((1378008790, 1.1421976184179525, 492688767), (1378008790, 0.376002688295385, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008788] : ((1378008788, -0.6075425510225507, 492688767), (1378008788, 0.49852935951611527, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008767] : ((1378008767, -3.2303396982863153, 492609224), (1378008767, 0.2954512998343307, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008759] : ((1378008759, -1.4454122245302994, 492688767), (1378008759, 0.6556355952402105, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008752] : ((1378008752, -0.8195509487313748, 492688767), (1378008752, 0.4790417877538337, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008733] : ((1378008733, -0.5969251627440688, 492688767), (1378008733, 0.5240104750565824, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008732] : ((1378008732, -0.9938047240144782, 492688767), (1378008732, 0.24549450633464245, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008731] : ((1378008731, -0.9625334989297292, 492688767), (1378008731, 0.16493148639094773, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008595] : ((1378008595, -1.4811292323886174, 492688767), (1378008595, 0.38259224702020506, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008570] : ((1378008570, -3.210455229956183, 492609224), (1378008570, 0.4066716262808126, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008515] : ((1378008515, 0.09787461520746246, 492688767), (1378008515, 0.36621690123629086, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008358] : ((1378008358, -2.176914757745654, 492609224), (1378008358, 0.7678348228580226, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008335] : ((1378008335, 1.4457476629545258, 492688767), (1378008335, 0.9488778463837493, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008295] : ((1378008295, -3.7716676713989634, 492609224), (1378008295, 0.41709811856832435, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008205] : ((1378008205, -3.455626636982021, 492609224), (1378008205, 0.2808324613375283, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008172] : ((1378008172, -3.9738477003262793, 492609224), (1378008172, 0.1933722744812085, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com args[1378008036] : ((1378008036, -0.6035043407521379, 492688767), (1378008036, 0.8785326100046361, 2107752395), '0.09801646468772696') We are sending mail with results at report@fotonower.com refus_total : 0.09801646468772696 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26075827 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26075827_20-08-2025_12_12_22.pdf results_Auto_P26075827_20-08-2025_12_12_22.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26075827_20-08-2025_12_12_22.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26075827','results_Auto_P26075827_20-08-2025_12_12_22.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26075827_20-08-2025_12_12_22.pdf','pdf','','0.38','0.09801646468772696') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26075827

https://www.fotonower.com/image?json=false&list_photos_id=1378008790
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.1421976184179525)
https://www.fotonower.com/image?json=false&list_photos_id=1378008788
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008767
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008759
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008752
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008733
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008732
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008731
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008595
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008570
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008515
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008358
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008335
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.4457476629545258)
https://www.fotonower.com/image?json=false&list_photos_id=1378008295
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008205
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008172
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1378008036
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 9.80%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/26075944?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/26075945?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/26075950?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26075951?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/26075953?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26075954?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26075827_20-08-2025_12_12_22.pdf.

Lien vers velours :https://www.fotonower.com/velours/26075944,26075945,26075946,26075947,26075948,26075949,26075950,26075951,26075952,26075953,26075954?tags=papier,pet_fonce,pehd,background,environnement,flou,metal,pet_clair,mal_croppe,autre,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 20 Aug 2025 10:12:27 GMT Content-Length: 0 Connection: close X-Message-Id: -UkRblqoQBO0AIDBY28t_g Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1378008790, 1378008788, 1378008767, 1378008759, 1378008752, 1378008733, 1378008732, 1378008731, 1378008595, 1378008570, 1378008515, 1378008358, 1378008335, 1378008295, 1378008205, 1378008172, 1378008036] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008790', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008788', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008767', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008759', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008752', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008733', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008732', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008731', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008595', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008570', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008515', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008358', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008335', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008295', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008205', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008172', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008036', None, None, None, None, None, '3556071') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 17 time used for this insertion : 0.017003774642944336 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.438906669616699 time spend to save output : 0.01735854148864746 total time spend for step 9 : 5.456265211105347 step10:split_time_score Wed Aug 20 12:12:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 17),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20082025 26075827 Nombre de photos uploadées : 17 / 23040 (0%) 20082025 26075827 Nombre de photos taguées (types de déchets): 0 / 17 (0%) 20082025 26075827 Nombre de photos taguées (volume) : 0 / 17 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 9.775161743164062e-06 ????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0008790493011474609 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21824359893798828 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.04855858571244855 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070736_20-08-2025_08_31_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26070736 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070736 AND mptpi.`type`=3594 To do Qualite : 0.12004560436795493 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070739_20-08-2025_08_22_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26070739 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070739 AND mptpi.`type`=3594 To do Qualite : 0.17909196566358027 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26070741_20-08-2025_08_11_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26070741 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26070741 AND mptpi.`type`=3594 To do Qualite : 0.18128038194444446 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26071862_20-08-2025_09_36_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26071862 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26071862 AND mptpi.`type`=3594 To do Qualite : 0.19035156250000002 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26071865_20-08-2025_09_11_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26071865 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26071865 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26075820 order by id desc limit 1 Qualite : 0.09801646468772696 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26075827_20-08-2025_12_12_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26075827 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26075827 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20082025': {'nb_upload': 17, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1378008790, 1378008788, 1378008767, 1378008759, 1378008752, 1378008733, 1378008732, 1378008731, 1378008595, 1378008570, 1378008515, 1378008358, 1378008335, 1378008295, 1378008205, 1378008172, 1378008036] Looping around the photos to save general results len do output : 1 /26075827Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008790', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008788', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008767', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008759', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008752', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008733', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008732', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008731', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008595', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008570', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008515', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008358', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008335', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008295', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008205', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008172', None, None, None, None, None, '3556071') ('3318', None, None, None, None, None, None, None, '3556071') ('3318', '26075827', '1378008036', None, None, None, None, None, '3556071') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.018763303756713867 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.8119101524353027 time spend to save output : 0.019086837768554688 total time spend for step 10 : 1.8309969902038574 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 17 set_done_treatment 64.22user 26.54system 2:03.92elapsed 73%CPU (0avgtext+0avgdata 3073680maxresident)k 499104inputs+33032outputs (55major+2166667minor)pagefaults 0swaps