python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3521114 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3594380'] with mtr_portfolio_ids : ['26218754'] and first list_photo_ids : [] new path : /proc/3521114/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 13 ; length of list_pids : 13 ; length of list_args : 13 time to download the photos : 2.1063413619995117 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Aug 26 01:30:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10609 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-26 01:30:33.040474: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-26 01:30:33.072530: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-26 01:30:33.074479: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f00e0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-26 01:30:33.074527: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-26 01:30:33.078162: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-26 01:30:33.231687: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3df49aa0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-26 01:30:33.231737: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-26 01:30:33.233141: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-26 01:30:33.233523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-26 01:30:33.236646: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-26 01:30:33.239289: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-26 01:30:33.239790: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-26 01:30:33.245276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-26 01:30:33.246278: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-26 01:30:33.251277: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-26 01:30:33.253026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-26 01:30:33.253106: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-26 01:30:33.254034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-26 01:30:33.254053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-26 01:30:33.254075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-26 01:30:33.255728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9827 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-26 01:30:33.499428: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-26 01:30:33.499527: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-26 01:30:33.499557: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-26 01:30:33.499583: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-26 01:30:33.499609: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-26 01:30:33.499635: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-26 01:30:33.499661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-26 01:30:33.499686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-26 01:30:33.501797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-26 01:30:33.502952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-26 01:30:33.502984: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-26 01:30:33.502999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-26 01:30:33.503014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-26 01:30:33.503027: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-26 01:30:33.503041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-26 01:30:33.503055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-26 01:30:33.503069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-26 01:30:33.504361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-26 01:30:33.504389: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-26 01:30:33.504397: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-26 01:30:33.504405: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-26 01:30:33.505723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9827 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-26 01:30:41.136676: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-26 01:30:41.347805: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 13 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 33.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 Detection mask done ! Trying to reset tf kernel 3521725 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 741 tf kernel not reseted sub process len(results) : 13 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 13 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1540 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0002815723419189453 nb_pixel_total : 7912 time to create 1 rle with old method : 0.009488821029663086 length of segment : 76 time for calcul the mask position with numpy : 0.00036644935607910156 nb_pixel_total : 11140 time to create 1 rle with old method : 0.016355514526367188 length of segment : 178 time for calcul the mask position with numpy : 0.00012803077697753906 nb_pixel_total : 4650 time to create 1 rle with old method : 0.0056531429290771484 length of segment : 107 time for calcul the mask position with numpy : 0.00011587142944335938 nb_pixel_total : 3595 time to create 1 rle with old method : 0.0045013427734375 length of segment : 66 time for calcul the mask position with numpy : 0.0004887580871582031 nb_pixel_total : 26898 time to create 1 rle with old method : 0.03112173080444336 length of segment : 288 time for calcul the mask position with numpy : 0.0001971721649169922 nb_pixel_total : 11770 time to create 1 rle with old method : 0.015477180480957031 length of segment : 131 time for calcul the mask position with numpy : 0.00010704994201660156 nb_pixel_total : 4514 time to create 1 rle with old method : 0.005340576171875 length of segment : 92 time for calcul the mask position with numpy : 0.0003249645233154297 nb_pixel_total : 18630 time to create 1 rle with old method : 0.021910667419433594 length of segment : 216 time for calcul the mask position with numpy : 0.0003254413604736328 nb_pixel_total : 22360 time to create 1 rle with old method : 0.025617599487304688 length of segment : 115 time for calcul the mask position with numpy : 0.00012540817260742188 nb_pixel_total : 3613 time to create 1 rle with old method : 0.0043871402740478516 length of segment : 81 time for calcul the mask position with numpy : 0.00014925003051757812 nb_pixel_total : 10348 time to create 1 rle with old method : 0.012206554412841797 length of segment : 73 time for calcul the mask position with numpy : 0.00024127960205078125 nb_pixel_total : 11337 time to create 1 rle with old method : 0.012978076934814453 length of segment : 170 time for calcul the mask position with numpy : 0.0002963542938232422 nb_pixel_total : 18308 time to create 1 rle with old method : 0.021216869354248047 length of segment : 199 time for calcul the mask position with numpy : 0.0003917217254638672 nb_pixel_total : 22199 time to create 1 rle with old method : 0.025666236877441406 length of segment : 187 time for calcul the mask position with numpy : 0.00010895729064941406 nb_pixel_total : 3484 time to create 1 rle with old method : 0.0045130252838134766 length of segment : 88 time for calcul the mask position with numpy : 0.002020597457885742 nb_pixel_total : 108381 time to create 1 rle with old method : 0.12008142471313477 length of segment : 521 time for calcul the mask position with numpy : 0.0002465248107910156 nb_pixel_total : 12313 time to create 1 rle with old method : 0.01399850845336914 length of segment : 176 time for calcul the mask position with numpy : 0.00012063980102539062 nb_pixel_total : 5361 time to create 1 rle with old method : 0.006386280059814453 length of segment : 119 time for calcul the mask position with numpy : 9.775161743164062e-05 nb_pixel_total : 5425 time to create 1 rle with old method : 0.006162405014038086 length of segment : 92 time for calcul the mask position with numpy : 0.0002315044403076172 nb_pixel_total : 12784 time to create 1 rle with old method : 0.014570236206054688 length of segment : 181 time for calcul the mask position with numpy : 0.00012373924255371094 nb_pixel_total : 7051 time to create 1 rle with old method : 0.008069515228271484 length of segment : 129 time for calcul the mask position with numpy : 0.0003676414489746094 nb_pixel_total : 28814 time to create 1 rle with old method : 0.033368825912475586 length of segment : 150 time for calcul the mask position with numpy : 0.00023126602172851562 nb_pixel_total : 11534 time to create 1 rle with old method : 0.013155698776245117 length of segment : 174 time for calcul the mask position with numpy : 0.00011968612670898438 nb_pixel_total : 5761 time to create 1 rle with old method : 0.0066776275634765625 length of segment : 77 time for calcul the mask position with numpy : 0.001367330551147461 nb_pixel_total : 98022 time to create 1 rle with old method : 0.10746097564697266 length of segment : 584 time for calcul the mask position with numpy : 0.000232696533203125 nb_pixel_total : 14076 time to create 1 rle with old method : 0.016213417053222656 length of segment : 169 time for calcul the mask position with numpy : 0.00016808509826660156 nb_pixel_total : 5961 time to create 1 rle with old method : 0.00735163688659668 length of segment : 152 time for calcul the mask position with numpy : 0.0003604888916015625 nb_pixel_total : 21893 time to create 1 rle with old method : 0.025267601013183594 length of segment : 229 time for calcul the mask position with numpy : 0.000148773193359375 nb_pixel_total : 7905 time to create 1 rle with old method : 0.00908041000366211 length of segment : 172 time for calcul the mask position with numpy : 0.00021910667419433594 nb_pixel_total : 7205 time to create 1 rle with old method : 0.008986949920654297 length of segment : 146 time for calcul the mask position with numpy : 0.008734941482543945 nb_pixel_total : 525398 time to create 1 rle with new method : 0.03427457809448242 length of segment : 951 time for calcul the mask position with numpy : 0.00015425682067871094 nb_pixel_total : 8199 time to create 1 rle with old method : 0.009772300720214844 length of segment : 129 time for calcul the mask position with numpy : 0.0001423358917236328 nb_pixel_total : 7597 time to create 1 rle with old method : 0.009217977523803711 length of segment : 84 time for calcul the mask position with numpy : 0.00013208389282226562 nb_pixel_total : 7515 time to create 1 rle with old method : 0.008660078048706055 length of segment : 93 time for calcul the mask position with numpy : 0.00022482872009277344 nb_pixel_total : 12437 time to create 1 rle with old method : 0.013983964920043945 length of segment : 178 time for calcul the mask position with numpy : 0.00011658668518066406 nb_pixel_total : 7159 time to create 1 rle with old method : 0.008301496505737305 length of segment : 89 time for calcul the mask position with numpy : 0.00016951560974121094 nb_pixel_total : 10238 time to create 1 rle with old method : 0.011880159378051758 length of segment : 121 time for calcul the mask position with numpy : 0.00013756752014160156 nb_pixel_total : 7927 time to create 1 rle with old method : 0.00922083854675293 length of segment : 112 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 8136 time to create 1 rle with old method : 0.009235382080078125 length of segment : 103 time for calcul the mask position with numpy : 0.0001556873321533203 nb_pixel_total : 4715 time to create 1 rle with old method : 0.0057811737060546875 length of segment : 94 time for calcul the mask position with numpy : 0.0001385211944580078 nb_pixel_total : 6738 time to create 1 rle with old method : 0.007716178894042969 length of segment : 108 time for calcul the mask position with numpy : 0.00019979476928710938 nb_pixel_total : 13354 time to create 1 rle with old method : 0.015156984329223633 length of segment : 145 time for calcul the mask position with numpy : 0.0013785362243652344 nb_pixel_total : 94993 time to create 1 rle with old method : 0.10479307174682617 length of segment : 516 time for calcul the mask position with numpy : 0.011563301086425781 nb_pixel_total : 702840 time to create 1 rle with new method : 0.03603863716125488 length of segment : 935 time for calcul the mask position with numpy : 0.0005662441253662109 nb_pixel_total : 9756 time to create 1 rle with old method : 0.010841131210327148 length of segment : 174 time for calcul the mask position with numpy : 0.0003986358642578125 nb_pixel_total : 10526 time to create 1 rle with old method : 0.011747360229492188 length of segment : 140 time for calcul the mask position with numpy : 0.00020170211791992188 nb_pixel_total : 7291 time to create 1 rle with old method : 0.008187294006347656 length of segment : 104 time for calcul the mask position with numpy : 0.0005075931549072266 nb_pixel_total : 10519 time to create 1 rle with old method : 0.011530160903930664 length of segment : 170 time for calcul the mask position with numpy : 0.0004131793975830078 nb_pixel_total : 12162 time to create 1 rle with old method : 0.014807462692260742 length of segment : 158 time for calcul the mask position with numpy : 0.0005080699920654297 nb_pixel_total : 11835 time to create 1 rle with old method : 0.016356706619262695 length of segment : 180 time for calcul the mask position with numpy : 0.02924323081970215 nb_pixel_total : 1217392 time to create 1 rle with new method : 0.7562117576599121 length of segment : 1960 time for calcul the mask position with numpy : 0.0015225410461425781 nb_pixel_total : 98102 time to create 1 rle with old method : 0.10603642463684082 length of segment : 530 time for calcul the mask position with numpy : 0.0004429817199707031 nb_pixel_total : 6642 time to create 1 rle with old method : 0.0073201656341552734 length of segment : 182 time for calcul the mask position with numpy : 0.0001811981201171875 nb_pixel_total : 3761 time to create 1 rle with old method : 0.004350900650024414 length of segment : 73 time for calcul the mask position with numpy : 0.0005950927734375 nb_pixel_total : 23210 time to create 1 rle with old method : 0.025462627410888672 length of segment : 145 time for calcul the mask position with numpy : 0.012100696563720703 nb_pixel_total : 660468 time to create 1 rle with new method : 0.0237884521484375 length of segment : 916 time for calcul the mask position with numpy : 0.00030159950256347656 nb_pixel_total : 10358 time to create 1 rle with old method : 0.011373043060302734 length of segment : 116 time for calcul the mask position with numpy : 0.0002484321594238281 nb_pixel_total : 5359 time to create 1 rle with old method : 0.005949735641479492 length of segment : 111 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 12956 time to create 1 rle with old method : 0.014221668243408203 length of segment : 191 time for calcul the mask position with numpy : 0.00023674964904785156 nb_pixel_total : 5752 time to create 1 rle with old method : 0.006691694259643555 length of segment : 83 time for calcul the mask position with numpy : 0.00031948089599609375 nb_pixel_total : 7533 time to create 1 rle with old method : 0.008626461029052734 length of segment : 117 time for calcul the mask position with numpy : 0.0010828971862792969 nb_pixel_total : 26390 time to create 1 rle with old method : 0.028757333755493164 length of segment : 394 time for calcul the mask position with numpy : 0.00011563301086425781 nb_pixel_total : 5154 time to create 1 rle with old method : 0.005791425704956055 length of segment : 98 time for calcul the mask position with numpy : 0.00067901611328125 nb_pixel_total : 25468 time to create 1 rle with old method : 0.027909040451049805 length of segment : 173 time spent for convertir_results : 4.3466455936431885 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 64 chid ids of type : 3594 Number RLEs to save : 14811 save missing photos in datou_result : time spend for datou_step_exec : 27.45554256439209 time spend to save output : 1.8046071529388428 total time spend for step 1 : 29.260149717330933 step2:crop_condition Tue Aug 26 01:30:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 13 ! batch 1 Loaded 64 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 35 About to insert : list_path_to_insert length 35 new photo from crops ! About to upload 35 photos upload in portfolio : 3736932 init cache_photo without model_param we have 35 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164662_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 35 photos in the portfolio 3736932 time of upload the photos Elapsed time : 9.437823295593262 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3736932 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164680_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.9267685413360596 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164683_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6108973026275635 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! About to upload 15 photos upload in portfolio : 3736932 init cache_photo without model_param we have 15 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164692_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 15 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.061039447784424 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164698_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9332621097564697 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164699_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.5695281028747559 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756164700_3521114 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6622545719146729 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1379227732, 1379227721, 1379227720, 1379227700, 1379227698, 1379227695, 1379227693, 1379227689, 1379227686, 1379227611, 1379227601, 1379227600, 1379227597] Looping around the photos to save general results len do output : 64 /1379351953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351976Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351978Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351981Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379351999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379352032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227732', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227721', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227720', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227700', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227698', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227695', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227693', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227689', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227686', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227611', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227601', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227600', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227597', None, None, None, None, None, '3594380') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 205 time used for this insertion : 1.6484510898590088 save_final save missing photos in datou_result : time spend for datou_step_exec : 41.26190948486328 time spend to save output : 1.6509151458740234 total time spend for step 2 : 42.912824630737305 step3:rle_unique_nms_with_priority Tue Aug 26 01:31:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 64 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 1.8451621532440186 time for calcul the mask position with numpy : 0.5490913391113281 nb_pixel_total : 2003121 time to create 1 rle with new method : 0.7256312370300293 time for calcul the mask position with numpy : 0.007544517517089844 nb_pixel_total : 4514 time to create 1 rle with old method : 0.005268573760986328 time for calcul the mask position with numpy : 0.008479595184326172 nb_pixel_total : 11770 time to create 1 rle with old method : 0.013519048690795898 time for calcul the mask position with numpy : 0.0074574947357177734 nb_pixel_total : 26898 time to create 1 rle with old method : 0.03077864646911621 time for calcul the mask position with numpy : 0.00691986083984375 nb_pixel_total : 3595 time to create 1 rle with old method : 0.0041997432708740234 time for calcul the mask position with numpy : 0.006524562835693359 nb_pixel_total : 4650 time to create 1 rle with old method : 0.00554347038269043 time for calcul the mask position with numpy : 0.006682395935058594 nb_pixel_total : 11140 time to create 1 rle with old method : 0.012923479080200195 time for calcul the mask position with numpy : 0.008533716201782227 nb_pixel_total : 7912 time to create 1 rle with old method : 0.009132146835327148 create new chi : 1.4189484119415283 time to delete rle : 0.01377105712890625 batch 1 Loaded 15 chid ids of type : 3594 ++++++++++Number RLEs to save : 2956 TO DO : save crop sub photo not yet done ! save time : 0.5188238620758057 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.38495397567749023 time for calcul the mask position with numpy : 0.07555413246154785 nb_pixel_total : 1989004 time to create 1 rle with new method : 0.08780145645141602 time for calcul the mask position with numpy : 0.007147073745727539 nb_pixel_total : 18308 time to create 1 rle with old method : 0.02128148078918457 time for calcul the mask position with numpy : 0.0067331790924072266 nb_pixel_total : 11337 time to create 1 rle with old method : 0.012977838516235352 time for calcul the mask position with numpy : 0.006956338882446289 nb_pixel_total : 10348 time to create 1 rle with old method : 0.014422416687011719 time for calcul the mask position with numpy : 0.006425380706787109 nb_pixel_total : 3613 time to create 1 rle with old method : 0.004301786422729492 time for calcul the mask position with numpy : 0.007078409194946289 nb_pixel_total : 22360 time to create 1 rle with old method : 0.025956153869628906 time for calcul the mask position with numpy : 0.007081270217895508 nb_pixel_total : 18630 time to create 1 rle with old method : 0.021544933319091797 create new chi : 0.31348085403442383 time to delete rle : 0.0003342628479003906 batch 1 Loaded 13 chid ids of type : 3594 ++++++Number RLEs to save : 2788 TO DO : save crop sub photo not yet done ! save time : 0.5305054187774658 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.05031609535217285 time for calcul the mask position with numpy : 0.01940178871154785 nb_pixel_total : 1939536 time to create 1 rle with new method : 0.08167362213134766 time for calcul the mask position with numpy : 0.007454872131347656 nb_pixel_total : 108381 time to create 1 rle with old method : 0.13457965850830078 time for calcul the mask position with numpy : 0.006361961364746094 nb_pixel_total : 3484 time to create 1 rle with old method : 0.004090070724487305 time for calcul the mask position with numpy : 0.006329536437988281 nb_pixel_total : 22199 time to create 1 rle with old method : 0.025524377822875977 create new chi : 0.28600168228149414 time to delete rle : 0.0004165172576904297 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 2672 TO DO : save crop sub photo not yet done ! save time : 0.19559121131896973 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.06268548965454102 time for calcul the mask position with numpy : 0.021031618118286133 nb_pixel_total : 2050501 time to create 1 rle with new method : 0.11977529525756836 time for calcul the mask position with numpy : 0.0063130855560302734 nb_pixel_total : 5425 time to create 1 rle with old method : 0.0061798095703125 time for calcul the mask position with numpy : 0.0064525604248046875 nb_pixel_total : 5361 time to create 1 rle with old method : 0.006137371063232422 time for calcul the mask position with numpy : 0.006340742111206055 nb_pixel_total : 12313 time to create 1 rle with old method : 0.01398777961730957 create new chi : 0.1866607666015625 time to delete rle : 0.0003845691680908203 batch 1 Loaded 7 chid ids of type : 3594 ++++Number RLEs to save : 1854 TO DO : save crop sub photo not yet done ! save time : 0.257554292678833 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.090606689453125 time for calcul the mask position with numpy : 0.08588361740112305 nb_pixel_total : 2053765 time to create 1 rle with new method : 0.22919607162475586 time for calcul the mask position with numpy : 0.006483554840087891 nb_pixel_total : 7051 time to create 1 rle with old method : 0.007947444915771484 time for calcul the mask position with numpy : 0.006838083267211914 nb_pixel_total : 12784 time to create 1 rle with old method : 0.014499187469482422 create new chi : 0.3618276119232178 time to delete rle : 0.0003063678741455078 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1700 TO DO : save crop sub photo not yet done ! save time : 0.30286526679992676 nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 0.5140419006347656 time for calcul the mask position with numpy : 0.052663326263427734 nb_pixel_total : 1879634 time to create 1 rle with new method : 0.08671307563781738 time for calcul the mask position with numpy : 0.006299495697021484 nb_pixel_total : 7905 time to create 1 rle with old method : 0.008807897567749023 time for calcul the mask position with numpy : 0.006255388259887695 nb_pixel_total : 21893 time to create 1 rle with old method : 0.02459096908569336 time for calcul the mask position with numpy : 0.006377220153808594 nb_pixel_total : 5961 time to create 1 rle with old method : 0.006996631622314453 time for calcul the mask position with numpy : 0.0069580078125 nb_pixel_total : 14076 time to create 1 rle with old method : 0.016208410263061523 time for calcul the mask position with numpy : 0.007104396820068359 nb_pixel_total : 98022 time to create 1 rle with old method : 0.11074113845825195 time for calcul the mask position with numpy : 0.006445646286010742 nb_pixel_total : 5761 time to create 1 rle with old method : 0.0065691471099853516 time for calcul the mask position with numpy : 0.006237030029296875 nb_pixel_total : 11534 time to create 1 rle with old method : 0.013089418411254883 time for calcul the mask position with numpy : 0.00633549690246582 nb_pixel_total : 28814 time to create 1 rle with old method : 0.032537221908569336 create new chi : 0.4203755855560303 time to delete rle : 0.0007038116455078125 batch 1 Loaded 17 chid ids of type : 3594 ++++++++++Number RLEs to save : 4494 TO DO : save crop sub photo not yet done ! save time : 0.29164886474609375 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 0.07615375518798828 time for calcul the mask position with numpy : 0.017571210861206055 nb_pixel_total : 1517686 time to create 1 rle with new method : 0.12045407295227051 time for calcul the mask position with numpy : 0.006175041198730469 nb_pixel_total : 7515 time to create 1 rle with old method : 0.008587837219238281 time for calcul the mask position with numpy : 0.006182193756103516 nb_pixel_total : 7597 time to create 1 rle with old method : 0.008666515350341797 time for calcul the mask position with numpy : 0.006121158599853516 nb_pixel_total : 8199 time to create 1 rle with old method : 0.009620904922485352 time for calcul the mask position with numpy : 0.020614147186279297 nb_pixel_total : 525398 time to create 1 rle with new method : 0.11975836753845215 time for calcul the mask position with numpy : 0.006570100784301758 nb_pixel_total : 7205 time to create 1 rle with old method : 0.008213520050048828 create new chi : 0.3575172424316406 time to delete rle : 0.0006561279296875 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 3886 TO DO : save crop sub photo not yet done ! save time : 0.7522132396697998 nb_obj : 7 nb_hashtags : 5 time to prepare the origin masks : 0.23502302169799805 time for calcul the mask position with numpy : 0.11266064643859863 nb_pixel_total : 2016250 time to create 1 rle with new method : 0.1082003116607666 time for calcul the mask position with numpy : 0.0063626766204833984 nb_pixel_total : 6738 time to create 1 rle with old method : 0.0077533721923828125 time for calcul the mask position with numpy : 0.0063800811767578125 nb_pixel_total : 4715 time to create 1 rle with old method : 0.005425930023193359 time for calcul the mask position with numpy : 0.006644487380981445 nb_pixel_total : 8136 time to create 1 rle with old method : 0.014174461364746094 time for calcul the mask position with numpy : 0.006619930267333984 nb_pixel_total : 7927 time to create 1 rle with old method : 0.009627819061279297 time for calcul the mask position with numpy : 0.006802082061767578 nb_pixel_total : 10238 time to create 1 rle with old method : 0.011566638946533203 time for calcul the mask position with numpy : 0.0068242549896240234 nb_pixel_total : 7159 time to create 1 rle with old method : 0.008159875869750977 time for calcul the mask position with numpy : 0.00672459602355957 nb_pixel_total : 12437 time to create 1 rle with old method : 0.014007091522216797 create new chi : 0.347902774810791 time to delete rle : 0.0005309581756591797 batch 1 Loaded 15 chid ids of type : 3594 +++++++Number RLEs to save : 2690 TO DO : save crop sub photo not yet done ! save time : 0.2037959098815918 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.05969047546386719 time for calcul the mask position with numpy : 0.01658153533935547 nb_pixel_total : 1262413 time to create 1 rle with new method : 0.031267404556274414 time for calcul the mask position with numpy : 0.011140584945678711 nb_pixel_total : 702840 time to create 1 rle with new method : 0.030483722686767578 time for calcul the mask position with numpy : 0.007323741912841797 nb_pixel_total : 94993 time to create 1 rle with old method : 0.1077733039855957 time for calcul the mask position with numpy : 0.006218433380126953 nb_pixel_total : 13354 time to create 1 rle with old method : 0.01539158821105957 create new chi : 0.22688031196594238 time to delete rle : 0.0007131099700927734 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 4272 TO DO : save crop sub photo not yet done ! save time : 0.27265048027038574 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.06028151512145996 time for calcul the mask position with numpy : 0.019901752471923828 nb_pixel_total : 2035508 time to create 1 rle with new method : 0.03003859519958496 time for calcul the mask position with numpy : 0.006339550018310547 nb_pixel_total : 10519 time to create 1 rle with old method : 0.011926651000976562 time for calcul the mask position with numpy : 0.006697177886962891 nb_pixel_total : 7291 time to create 1 rle with old method : 0.008307218551635742 time for calcul the mask position with numpy : 0.006209850311279297 nb_pixel_total : 10526 time to create 1 rle with old method : 0.011905431747436523 time for calcul the mask position with numpy : 0.006739616394042969 nb_pixel_total : 9756 time to create 1 rle with old method : 0.011188745498657227 create new chi : 0.11974287033081055 time to delete rle : 0.0005333423614501953 batch 1 Loaded 9 chid ids of type : 3594 ++++++Number RLEs to save : 2256 TO DO : save crop sub photo not yet done ! save time : 0.550368070602417 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.532923698425293 time for calcul the mask position with numpy : 0.026877403259277344 nb_pixel_total : 738057 time to create 1 rle with new method : 0.10588860511779785 time for calcul the mask position with numpy : 0.006919145584106445 nb_pixel_total : 98102 time to create 1 rle with old method : 0.11231565475463867 time for calcul the mask position with numpy : 0.03704500198364258 nb_pixel_total : 1213444 time to create 1 rle with new method : 0.12772870063781738 time for calcul the mask position with numpy : 0.006662130355834961 nb_pixel_total : 11835 time to create 1 rle with old method : 0.013409137725830078 time for calcul the mask position with numpy : 0.0064542293548583984 nb_pixel_total : 12162 time to create 1 rle with old method : 0.014054059982299805 create new chi : 0.4728386402130127 time to delete rle : 0.0009560585021972656 batch 1 Loaded 9 chid ids of type : 3594 +++++++Number RLEs to save : 6603 TO DO : save crop sub photo not yet done ! save time : 0.4012322425842285 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.24907326698303223 time for calcul the mask position with numpy : 0.05484580993652344 nb_pixel_total : 1369161 time to create 1 rle with new method : 0.19432711601257324 time for calcul the mask position with numpy : 0.0064411163330078125 nb_pixel_total : 10358 time to create 1 rle with old method : 0.011909723281860352 time for calcul the mask position with numpy : 0.010981082916259766 nb_pixel_total : 660468 time to create 1 rle with new method : 0.07503986358642578 time for calcul the mask position with numpy : 0.006367683410644531 nb_pixel_total : 23210 time to create 1 rle with old method : 0.026163339614868164 time for calcul the mask position with numpy : 0.00635981559753418 nb_pixel_total : 3761 time to create 1 rle with old method : 0.004289865493774414 time for calcul the mask position with numpy : 0.006287097930908203 nb_pixel_total : 6642 time to create 1 rle with old method : 0.007683515548706055 create new chi : 0.420245885848999 time to delete rle : 0.0006549358367919922 batch 1 Loaded 11 chid ids of type : 3594 ++++++Number RLEs to save : 3944 TO DO : save crop sub photo not yet done ! save time : 0.5268075466156006 nb_obj : 7 nb_hashtags : 2 time to prepare the origin masks : 0.0947427749633789 time for calcul the mask position with numpy : 0.02011895179748535 nb_pixel_total : 1984988 time to create 1 rle with new method : 0.15003705024719238 time for calcul the mask position with numpy : 0.006295680999755859 nb_pixel_total : 25468 time to create 1 rle with old method : 0.029604673385620117 time for calcul the mask position with numpy : 0.006222724914550781 nb_pixel_total : 5154 time to create 1 rle with old method : 0.0060236454010009766 time for calcul the mask position with numpy : 0.006455898284912109 nb_pixel_total : 26390 time to create 1 rle with old method : 0.030374765396118164 time for calcul the mask position with numpy : 0.0062503814697265625 nb_pixel_total : 7533 time to create 1 rle with old method : 0.008701562881469727 time for calcul the mask position with numpy : 0.006371736526489258 nb_pixel_total : 5752 time to create 1 rle with old method : 0.006668567657470703 time for calcul the mask position with numpy : 0.006351947784423828 nb_pixel_total : 12956 time to create 1 rle with old method : 0.015041351318359375 time for calcul the mask position with numpy : 0.006780385971069336 nb_pixel_total : 5359 time to create 1 rle with old method : 0.006209373474121094 create new chi : 0.3277130126953125 time to delete rle : 0.0005381107330322266 batch 1 Loaded 15 chid ids of type : 3594 ++++++++++Number RLEs to save : 3414 TO DO : save crop sub photo not yet done ! save time : 0.22258615493774414 map_output_result : {1379227732: (0.0, 'Should be the crop_list due to order', 0), 1379227721: (0.0, 'Should be the crop_list due to order', 0), 1379227720: (0.0, 'Should be the crop_list due to order', 0), 1379227700: (0.0, 'Should be the crop_list due to order', 0), 1379227698: (0.0, 'Should be the crop_list due to order', 0), 1379227695: (0.0, 'Should be the crop_list due to order', 0), 1379227693: (0.0, 'Should be the crop_list due to order', 0), 1379227689: (0.0, 'Should be the crop_list due to order', 0), 1379227686: (0.0, 'Should be the crop_list due to order', 0), 1379227611: (0.0, 'Should be the crop_list due to order', 0), 1379227601: (0.0, 'Should be the crop_list due to order', 0), 1379227600: (0.0, 'Should be the crop_list due to order', 0), 1379227597: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1379227732, 1379227721, 1379227720, 1379227700, 1379227698, 1379227695, 1379227693, 1379227689, 1379227686, 1379227611, 1379227601, 1379227600, 1379227597] Looping around the photos to save general results len do output : 13 /1379227732.Didn't retrieve data . /1379227721.Didn't retrieve data . /1379227720.Didn't retrieve data . /1379227700.Didn't retrieve data . /1379227698.Didn't retrieve data . /1379227695.Didn't retrieve data . /1379227693.Didn't retrieve data . /1379227689.Didn't retrieve data . /1379227686.Didn't retrieve data . /1379227611.Didn't retrieve data . /1379227601.Didn't retrieve data . /1379227600.Didn't retrieve data . /1379227597.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227732', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227721', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227720', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227700', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227698', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227695', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227693', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227689', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227686', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227611', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227601', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227600', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227597', None, None, None, None, None, '3594380') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 39 time used for this insertion : 0.016771316528320312 save_final save missing photos in datou_result : time spend for datou_step_exec : 15.048408269882202 time spend to save output : 0.017304182052612305 total time spend for step 3 : 15.065712451934814 step4:ventilate_hashtags_in_portfolio Tue Aug 26 01:31:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26218754 get user id for portfolio 26218754 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218754 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','papier','pehd','pet_fonce','environnement','metal','mal_croppe','carton','flou','background','pet_clair')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218754 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre','papier','pehd','pet_fonce','environnement','metal','mal_croppe','carton','flou','background','pet_clair')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/26219824,26219825,26219826,26219827,26219828,26219829,26219830,26219831,26219832,26219833,26219834?tags=environnement,autre,pet_fonce,flou,carton,pet_clair,background,mal_croppe,papier,metal,pehd Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1379227732, 1379227721, 1379227720, 1379227700, 1379227698, 1379227695, 1379227693, 1379227689, 1379227686, 1379227611, 1379227601, 1379227600, 1379227597] Looping around the photos to save general results len do output : 1 /26218754. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227732', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227721', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227720', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227700', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227698', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227695', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227693', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227689', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227686', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227611', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227601', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227600', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227597', None, None, None, None, None, '3594380') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 14 time used for this insertion : 0.3676941394805908 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.01298189163208 time spend to save output : 0.3679924011230469 total time spend for step 4 : 1.380974292755127 step5:final Tue Aug 26 01:31:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1379227732: ('0.10144373835313303',), 1379227721: ('0.10144373835313303',), 1379227720: ('0.10144373835313303',), 1379227700: ('0.10144373835313303',), 1379227698: ('0.10144373835313303',), 1379227695: ('0.10144373835313303',), 1379227693: ('0.10144373835313303',), 1379227689: ('0.10144373835313303',), 1379227686: ('0.10144373835313303',), 1379227611: ('0.10144373835313303',), 1379227601: ('0.10144373835313303',), 1379227600: ('0.10144373835313303',), 1379227597: ('0.10144373835313303',)} new output for save of step final : {1379227732: ('0.10144373835313303',), 1379227721: ('0.10144373835313303',), 1379227720: ('0.10144373835313303',), 1379227700: ('0.10144373835313303',), 1379227698: ('0.10144373835313303',), 1379227695: ('0.10144373835313303',), 1379227693: ('0.10144373835313303',), 1379227689: ('0.10144373835313303',), 1379227686: ('0.10144373835313303',), 1379227611: ('0.10144373835313303',), 1379227601: ('0.10144373835313303',), 1379227600: ('0.10144373835313303',), 1379227597: ('0.10144373835313303',)} [1379227732, 1379227721, 1379227720, 1379227700, 1379227698, 1379227695, 1379227693, 1379227689, 1379227686, 1379227611, 1379227601, 1379227600, 1379227597] Looping around the photos to save general results len do output : 13 /1379227732.Didn't retrieve data . /1379227721.Didn't retrieve data . /1379227720.Didn't retrieve data . /1379227700.Didn't retrieve data . /1379227698.Didn't retrieve data . /1379227695.Didn't retrieve data . /1379227693.Didn't retrieve data . /1379227689.Didn't retrieve data . /1379227686.Didn't retrieve data . /1379227611.Didn't retrieve data . /1379227601.Didn't retrieve data . /1379227600.Didn't retrieve data . /1379227597.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227732', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227721', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227720', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227700', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227698', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227695', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227693', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227689', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227686', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227611', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227601', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227600', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227597', None, None, None, None, None, '3594380') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 39 time used for this insertion : 0.013361215591430664 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.14796066284179688 time spend to save output : 0.014077186584472656 total time spend for step 5 : 0.16203784942626953 step6:blur_detection Tue Aug 26 01:31:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c.jpg resize: (1080, 1920) 1379227732 0.7951303590771547 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e.jpg resize: (1080, 1920) 1379227721 -0.5339306916737872 treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b.jpg resize: (1080, 1920) 1379227720 1.8448622758367244 treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a.jpg resize: (1080, 1920) 1379227700 1.5197027677145136 treat image : temp/1756164628_3521114_1379227698_5cf4d354c676d5edea6ddd6fffab53c0.jpg resize: (1080, 1920) 1379227698 0.2637430971658594 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b.jpg resize: (1080, 1920) 1379227695 1.8313409239309855 treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767.jpg resize: (1080, 1920) 1379227693 0.17173247302802302 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a.jpg resize: (1080, 1920) 1379227689 0.5863718125861059 treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196.jpg resize: (1080, 1920) 1379227686 2.332090109689103 treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d.jpg resize: (1080, 1920) 1379227611 0.9884552217806717 treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378.jpg resize: (1080, 1920) 1379227601 -0.4609683070376912 treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9.jpg resize: (1080, 1920) 1379227600 -0.13494162791100975 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d.jpg resize: (1080, 1920) 1379227597 0.4943365455600388 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512847_0.png resize: (75, 136) 1379351953 0.022294039354897936 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512848_0.png resize: (178, 117) 1379351954 -0.7164282997961484 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512849_0.png resize: (107, 74) 1379351955 -1.7085430497844167 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512850_0.png resize: (75, 115) 1379351956 -1.0570931309465117 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512856_0.png resize: (69, 95) 1379351957 -1.064159815880347 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512858_0.png resize: (170, 124) 1379351958 -0.6015403502002874 treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b_rle_crop_3930512861_0.png resize: (50, 133) 1379351959 -2.7330256182341333 treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a_rle_crop_3930512863_0.png resize: (181, 119) 1379351960 -0.9807114547242498 treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a_rle_crop_3930512864_0.png resize: (88, 121) 1379351961 -1.6728758889877333 treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a_rle_crop_3930512865_0.png resize: (91, 77) 1379351962 0.878395868618114 treat image : temp/1756164628_3521114_1379227698_5cf4d354c676d5edea6ddd6fffab53c0_rle_crop_3930512866_0.png resize: (181, 130) 1379351963 -0.6172739234991009 treat image : temp/1756164628_3521114_1379227698_5cf4d354c676d5edea6ddd6fffab53c0_rle_crop_3930512867_0.png resize: (129, 60) 1379351964 6.791276005733303 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512869_0.png resize: (174, 119) 1379351965 -0.4909807424742669 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512870_0.png resize: (76, 99) 1379351966 -0.13994987237338247 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512873_0.png resize: (108, 133) 1379351967 -2.016986726393561 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512875_0.png resize: (172, 61) 1379351968 -0.6152808586435963 treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512876_0.png resize: (146, 89) 1379351969 -0.9059672075821499 treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512878_0.png resize: (129, 84) 1379351970 -0.8926198336114575 treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512879_0.png resize: (82, 137) 1379351971 -0.8324961455381964 treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512880_0.png resize: (92, 107) 1379351972 -1.019568266220233 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512881_0.png resize: (178, 127) 1379351973 -0.9434696943265679 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512882_0.png resize: (89, 110) 1379351974 -1.1831146140593005 treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196_rle_crop_3930512888_0.png resize: (139, 125) 1379351975 0.7689461239811654 treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512891_0.png resize: (174, 76) 1379351976 -0.11454179626413145 treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512892_0.png resize: (113, 159) 1379351977 -0.8138348481994987 treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512894_0.png resize: (171, 111) 1379351978 -0.7312795878301479 treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512895_0.png resize: (153, 112) 1379351979 -1.1750464008364188 treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512896_0.png resize: (180, 110) 1379351980 -0.5144679244550119 treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512900_0.png resize: (73, 83) 1379351981 -0.43151658104538176 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512904_0.png resize: (111, 60) 1379351982 0.6558777168783791 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512905_0.png resize: (191, 129) 1379351983 -0.5812524610454591 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512906_0.png resize: (80, 119) 1379351984 -0.8105440926839503 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512907_0.png resize: (120, 102) 1379351985 -1.6936960383511823 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512908_0.png resize: (340, 180) 1379351986 -1.1690981269236604 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512909_0.png resize: (96, 74) 1379351987 -0.5063052846267767 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512859_0.png resize: (198, 130) 1379351989 -0.7132355342976513 treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b_rle_crop_3930512860_0.png resize: (158, 197) 1379351990 -1.6193963440744192 treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512877_0.png resize: (950, 813) 1379351991 0.3537280408291946 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512883_0.png resize: (117, 113) 1379351992 -0.8990955733233936 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512886_0.png resize: (86, 104) 1379351993 -2.1324659622068425 treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196_rle_crop_3930512890_0.png resize: (934, 1029) 1379351994 0.9318956322582593 treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512893_0.png resize: (104, 102) 1379351995 -1.491250363315539 treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512899_0.png resize: (175, 101) 1379351996 -2.16173553244897 treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512902_0.png resize: (914, 952) 1379351998 0.3368066276108217 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512885_0.png resize: (93, 102) 1379351999 -0.7351089199831683 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512851_0.png resize: (206, 221) 1379352011 -1.6654521529956858 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512854_0.png resize: (204, 140) 1379352012 -2.8119298447741006 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512855_0.png resize: (113, 239) 1379352013 0.9818928876234984 treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512857_0.png resize: (73, 175) 1379352014 -1.7318528542147458 treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b_rle_crop_3930512862_0.png resize: (516, 347) 1379352015 -0.18836034278539293 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512868_0.png resize: (147, 304) 1379352016 0.018984027764478793 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512871_0.png resize: (523, 331) 1379352017 -0.5110647707533467 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512872_0.png resize: (167, 99) 1379352018 2.1584509299891432 treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512874_0.png resize: (229, 140) 1379352019 -1.1388730134202516 treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196_rle_crop_3930512889_0.png resize: (463, 342) 1379352020 -0.5185439972851741 treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512897_0.png resize: (986, 1613) 1379352021 -1.8951230176497766 treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512898_0.png resize: (529, 333) 1379352022 0.1716824617207402 treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512901_0.png resize: (145, 211) 1379352023 -1.4754220990776685 treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512903_0.png resize: (114, 116) 1379352024 -3.319731418559839 treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512910_0.png resize: (158, 243) 1379352025 -0.582874215009428 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512852_0.png resize: (130, 121) 1379352026 -0.8431000595858941 treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512853_0.png resize: (92, 74) 1379352027 -2.9142615854325373 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512884_0.png resize: (112, 95) 1379352029 -0.7436416442463796 treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512887_0.png resize: (108, 92) 1379352032 -1.1807440653813517 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 77 time used for this insertion : 0.01458740234375 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 77 time used for this insertion : 0.020874738693237305 save missing photos in datou_result : time spend for datou_step_exec : 12.855703830718994 time spend to save output : 0.04059028625488281 total time spend for step 6 : 12.896294116973877 step7:brightness Tue Aug 26 01:32:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c.jpg treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e.jpg treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b.jpg treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a.jpg treat image : temp/1756164628_3521114_1379227698_5cf4d354c676d5edea6ddd6fffab53c0.jpg treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b.jpg treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767.jpg treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a.jpg treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196.jpg treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d.jpg treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378.jpg treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9.jpg treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d.jpg treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512847_0.png treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512848_0.png treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512849_0.png treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512850_0.png treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512856_0.png treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512858_0.png treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b_rle_crop_3930512861_0.png treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a_rle_crop_3930512863_0.png treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a_rle_crop_3930512864_0.png treat image : temp/1756164628_3521114_1379227700_6bc6fbd0bbcd79581eb4675a815e6a0a_rle_crop_3930512865_0.png treat image : temp/1756164628_3521114_1379227698_5cf4d354c676d5edea6ddd6fffab53c0_rle_crop_3930512866_0.png treat image : temp/1756164628_3521114_1379227698_5cf4d354c676d5edea6ddd6fffab53c0_rle_crop_3930512867_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512869_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512870_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512873_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512875_0.png treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512876_0.png treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512878_0.png treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512879_0.png treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512880_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512881_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512882_0.png treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196_rle_crop_3930512888_0.png treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512891_0.png treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512892_0.png treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512894_0.png treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512895_0.png treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512896_0.png treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512900_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512904_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512905_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512906_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512907_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512908_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512909_0.png treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512859_0.png treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b_rle_crop_3930512860_0.png treat image : temp/1756164628_3521114_1379227693_da7cd280c2c9ae07d56465d2b14b4767_rle_crop_3930512877_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512883_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512886_0.png treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196_rle_crop_3930512890_0.png treat image : temp/1756164628_3521114_1379227611_1628999759ae90d2055f7624e5ffb41d_rle_crop_3930512893_0.png treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512899_0.png treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512902_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512885_0.png treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512851_0.png treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512854_0.png treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512855_0.png treat image : temp/1756164628_3521114_1379227721_3423727081753286411555af6b63f85e_rle_crop_3930512857_0.png treat image : temp/1756164628_3521114_1379227720_90e97b6241af1d4d053b2068c8d51d5b_rle_crop_3930512862_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512868_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512871_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512872_0.png treat image : temp/1756164628_3521114_1379227695_ccecec5be9549573e329f717fba6263b_rle_crop_3930512874_0.png treat image : temp/1756164628_3521114_1379227686_5d1db3d29745b87fd80bbfefd56ae196_rle_crop_3930512889_0.png treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512897_0.png treat image : temp/1756164628_3521114_1379227601_0fc723da97d56875a06de78dd821b378_rle_crop_3930512898_0.png treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512901_0.png treat image : temp/1756164628_3521114_1379227600_bfcf8e05a8f7a2de39783db337daebc9_rle_crop_3930512903_0.png treat image : temp/1756164628_3521114_1379227597_9adeb4dc0563341c11b633973128ce2d_rle_crop_3930512910_0.png treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512852_0.png treat image : temp/1756164628_3521114_1379227732_be823b188906adcaf8bf5ffa93319b0c_rle_crop_3930512853_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512884_0.png treat image : temp/1756164628_3521114_1379227689_0ad7d1f39032c12411bbbe969cb9e18a_rle_crop_3930512887_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 77 time used for this insertion : 0.014106273651123047 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 77 time used for this insertion : 0.12624025344848633 save missing photos in datou_result : time spend for datou_step_exec : 3.68807053565979 time spend to save output : 0.14588284492492676 total time spend for step 7 : 3.833953380584717 step8:velours_tree Tue Aug 26 01:32:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.08605766296386719 time spend to save output : 4.696846008300781e-05 total time spend for step 8 : 0.0861046314239502 step9:send_mail_cod Tue Aug 26 01:32:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26218754_26-08-2025_01_32_15.pdf 26219825 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262198251756164735 26219826 change filename to text .change filename to text .change filename to text .imagette262198261756164736 26219827 imagette262198271756164736 26219828 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262198281756164736 26219829 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262198291756164738 26219830 imagette262198301756164739 26219831 imagette262198311756164739 26219832 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262198321756164739 26219833 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262198331756164740 26219834 change filename to text .change filename to text .imagette262198341756164741 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26218754 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26219824,26219825,26219826,26219827,26219828,26219829,26219830,26219831,26219832,26219833,26219834?tags=environnement,autre,pet_fonce,flou,carton,pet_clair,background,mal_croppe,papier,metal,pehd args[1379227732] : ((1379227732, 0.7951303590771547, 492688767), (1379227732, 0.48568063434550274, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227721] : ((1379227721, -0.5339306916737872, 492688767), (1379227721, 0.723968572813972, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227720] : ((1379227720, 1.8448622758367244, 492688767), (1379227720, 0.8578560727588479, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227700] : ((1379227700, 1.5197027677145136, 492688767), (1379227700, 0.6900631732048225, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227698] : ((1379227698, 0.2637430971658594, 492688767), (1379227698, 0.5886885819747413, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227695] : ((1379227695, 1.8313409239309855, 492688767), (1379227695, 0.6736869353126056, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227693] : ((1379227693, 0.17173247302802302, 492688767), (1379227693, 0.46243391124133293, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227689] : ((1379227689, 0.5863718125861059, 492688767), (1379227689, 0.4449051998124987, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227686] : ((1379227686, 2.332090109689103, 492688767), (1379227686, 0.7970851897717715, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227611] : ((1379227611, 0.9884552217806717, 492688767), (1379227611, 0.5018603114623205, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227601] : ((1379227601, -0.4609683070376912, 492688767), (1379227601, 0.6285292108522189, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227600] : ((1379227600, -0.13494162791100975, 492688767), (1379227600, 0.631371229014911, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com args[1379227597] : ((1379227597, 0.4943365455600388, 492688767), (1379227597, 0.6806198455134865, 2107752395), '0.10144373835313303') We are sending mail with results at report@fotonower.com refus_total : 0.10144373835313303 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=26218754 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218754_26-08-2025_01_32_15.pdf results_Auto_P26218754_26-08-2025_01_32_15.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218754_26-08-2025_01_32_15.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','26218754','results_Auto_P26218754_26-08-2025_01_32_15.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218754_26-08-2025_01_32_15.pdf','pdf','','0.71','0.10144373835313303') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/26218754

https://www.fotonower.com/image?json=false&list_photos_id=1379227732
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227721
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227720
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.8448622758367244)
https://www.fotonower.com/image?json=false&list_photos_id=1379227700
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.5197027677145136)
https://www.fotonower.com/image?json=false&list_photos_id=1379227698
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227695
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.8313409239309855)
https://www.fotonower.com/image?json=false&list_photos_id=1379227693
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227689
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227686
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.332090109689103)
https://www.fotonower.com/image?json=false&list_photos_id=1379227611
La photo est trop floue, merci de reprendre une photo.(avec le score = 0.9884552217806717)
https://www.fotonower.com/image?json=false&list_photos_id=1379227601
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227600
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1379227597
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 10.14%
Veuillez trouver les photos des contaminants.

exemples de contaminants: autre: https://www.fotonower.com/view/26219825?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/26219826?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/26219828?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/26219829?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/26219832?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/26219833?limit=200
exemples de contaminants: pehd: https://www.fotonower.com/view/26219834?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218754_26-08-2025_01_32_15.pdf.

Lien vers velours :https://www.fotonower.com/velours/26219824,26219825,26219826,26219827,26219828,26219829,26219830,26219831,26219832,26219833,26219834?tags=environnement,autre,pet_fonce,flou,carton,pet_clair,background,mal_croppe,papier,metal,pehd.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 25 Aug 2025 23:32:22 GMT Content-Length: 0 Connection: close X-Message-Id: nBF_PTqeT7i8fIvCW8d9TQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1379227732, 1379227721, 1379227720, 1379227700, 1379227698, 1379227695, 1379227693, 1379227689, 1379227686, 1379227611, 1379227601, 1379227600, 1379227597] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227732', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227721', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227720', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227700', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227698', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227695', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227693', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227689', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227686', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227611', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227601', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227600', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227597', None, None, None, None, None, '3594380') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 13 time used for this insertion : 0.014778375625610352 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.180785655975342 time spend to save output : 0.015038490295410156 total time spend for step 9 : 7.195824146270752 step10:split_time_score Tue Aug 26 01:32:23 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('14', 53),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 25082025 26218754 Nombre de photos uploadées : 53 / 23040 (0%) 25082025 26218754 Nombre de photos taguées (types de déchets): 0 / 53 (0%) 25082025 26218754 Nombre de photos taguées (volume) : 0 / 53 (0%) elapsed_time : load_data_split_time_score 1.1920928955078125e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0023953914642333984 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21841192245483398 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.1956718898385565 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26196500_25-08-2025_11_02_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26196500 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26196500 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218736 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218739 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218741 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218742 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218744 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218745 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218748 order by id desc limit 1 Qualite : 0.08477493248456791 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218754_26-08-2025_01_32_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218754 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218754 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218755 order by id desc limit 1 Qualite : 0.07988266411443495 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218757_26-08-2025_01_11_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218757 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218757 AND mptpi.`type`=3594 To do Qualite : 0.019049177758487655 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218778_26-08-2025_01_01_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218778 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218778 AND mptpi.`type`=3594 To do Qualite : 0.14097331175697297 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218787_26-08-2025_00_53_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218787 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218787 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218819 order by id desc limit 1 Qualite : 0.30717133403784214 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218840_26-08-2025_00_33_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218840 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218840 AND mptpi.`type`=3594 To do Qualite : 0.14214023919753085 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26218845_26-08-2025_00_21_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26218845 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26218845 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'25082025': {'nb_upload': 53, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1379227732, 1379227721, 1379227720, 1379227700, 1379227698, 1379227695, 1379227693, 1379227689, 1379227686, 1379227611, 1379227601, 1379227600, 1379227597] Looping around the photos to save general results len do output : 1 /26218754Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227732', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227721', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227720', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227700', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227698', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227695', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227693', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227689', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227686', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227611', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227601', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227600', None, None, None, None, None, '3594380') ('3318', None, None, None, None, None, None, None, '3594380') ('3318', None, '1379227597', None, None, None, None, None, '3594380') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 14 time used for this insertion : 0.8480455875396729 save_final save missing photos in datou_result : time spend for datou_step_exec : 9.355921268463135 time spend to save output : 0.8483970165252686 total time spend for step 10 : 10.204318284988403 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 13 set_done_treatment 60.84user 23.29system 2:07.26elapsed 66%CPU (0avgtext+0avgdata 2914008maxresident)k 56096inputs+42120outputs (40major+1884164minor)pagefaults 0swaps