python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1007243 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['4053083'] with mtr_portfolio_ids : ['28565337'] and first list_photo_ids : [] new path : /proc/1007243/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 17 ; length of list_pids : 17 ; length of list_args : 17 time to download the photos : 2.3245201110839844 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Sat Nov 15 09:50:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-11-15 09:50:32.880407: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-11-15 09:50:32.906430: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz 2025-11-15 09:50:32.908172: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f558c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-11-15 09:50:32.908213: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-11-15 09:50:32.911290: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-11-15 09:50:33.233958: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xbe3d560 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-11-15 09:50:33.234001: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-11-15 09:50:33.235398: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-11-15 09:50:33.235789: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-15 09:50:33.238660: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-15 09:50:33.241362: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-11-15 09:50:33.241924: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-11-15 09:50:33.245050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-11-15 09:50:33.246322: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-11-15 09:50:33.251443: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-11-15 09:50:33.253256: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-11-15 09:50:33.253332: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-15 09:50:33.254286: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-11-15 09:50:33.254306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-11-15 09:50:33.254317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-11-15 09:50:33.256047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-11-15 09:50:33.559735: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-11-15 09:50:33.559811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-15 09:50:33.559827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-15 09:50:33.559841: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-11-15 09:50:33.559855: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-11-15 09:50:33.559868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-11-15 09:50:33.559881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-11-15 09:50:33.559895: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-11-15 09:50:33.561112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-11-15 09:50:33.562231: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-11-15 09:50:33.562266: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-11-15 09:50:33.562297: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-15 09:50:33.562315: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-11-15 09:50:33.562331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-11-15 09:50:33.562347: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-11-15 09:50:33.562363: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-11-15 09:50:33.562379: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-11-15 09:50:33.563684: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-11-15 09:50:33.563712: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-11-15 09:50:33.563721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-11-15 09:50:33.563729: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-11-15 09:50:33.565117: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-11-15 09:50:41.904713: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-11-15 09:50:42.086376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 17 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 19.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 15.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 1 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 17.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 46.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 43.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 26.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 44.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 43.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 1007835 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 17 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 17 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0002608299255371094 nb_pixel_total : 9302 time to create 1 rle with old method : 0.010014057159423828 length of segment : 142 time for calcul the mask position with numpy : 0.0007221698760986328 nb_pixel_total : 36487 time to create 1 rle with old method : 0.038909196853637695 length of segment : 205 time for calcul the mask position with numpy : 0.00014734268188476562 nb_pixel_total : 6481 time to create 1 rle with old method : 0.006924867630004883 length of segment : 100 time for calcul the mask position with numpy : 0.0002040863037109375 nb_pixel_total : 12402 time to create 1 rle with old method : 0.013140678405761719 length of segment : 120 time for calcul the mask position with numpy : 0.02717900276184082 nb_pixel_total : 713681 time to create 1 rle with new method : 0.05251026153564453 length of segment : 979 time for calcul the mask position with numpy : 0.0015370845794677734 nb_pixel_total : 100990 time to create 1 rle with old method : 0.10841655731201172 length of segment : 534 time for calcul the mask position with numpy : 8.797645568847656e-05 nb_pixel_total : 3086 time to create 1 rle with old method : 0.0033440589904785156 length of segment : 69 time for calcul the mask position with numpy : 9.298324584960938e-05 nb_pixel_total : 3331 time to create 1 rle with old method : 0.0036373138427734375 length of segment : 80 time for calcul the mask position with numpy : 0.0015385150909423828 nb_pixel_total : 110030 time to create 1 rle with old method : 0.11477494239807129 length of segment : 535 time for calcul the mask position with numpy : 0.0002338886260986328 nb_pixel_total : 13186 time to create 1 rle with old method : 0.014255762100219727 length of segment : 155 time for calcul the mask position with numpy : 7.867813110351562e-05 nb_pixel_total : 1827 time to create 1 rle with old method : 0.00217437744140625 length of segment : 44 time for calcul the mask position with numpy : 9.250640869140625e-05 nb_pixel_total : 2633 time to create 1 rle with old method : 0.0031049251556396484 length of segment : 66 time for calcul the mask position with numpy : 0.0002834796905517578 nb_pixel_total : 17920 time to create 1 rle with old method : 0.019785165786743164 length of segment : 165 time for calcul the mask position with numpy : 0.0016028881072998047 nb_pixel_total : 105094 time to create 1 rle with old method : 0.10986542701721191 length of segment : 525 time for calcul the mask position with numpy : 0.00011754035949707031 nb_pixel_total : 5468 time to create 1 rle with old method : 0.0059931278228759766 length of segment : 66 time for calcul the mask position with numpy : 0.0001685619354248047 nb_pixel_total : 9890 time to create 1 rle with old method : 0.010889053344726562 length of segment : 92 time for calcul the mask position with numpy : 0.0001494884490966797 nb_pixel_total : 7476 time to create 1 rle with old method : 0.01372528076171875 length of segment : 126 time for calcul the mask position with numpy : 0.0004343986511230469 nb_pixel_total : 20355 time to create 1 rle with old method : 0.022791624069213867 length of segment : 130 time for calcul the mask position with numpy : 0.025855541229248047 nb_pixel_total : 1233838 time to create 1 rle with new method : 0.07024502754211426 length of segment : 1015 time for calcul the mask position with numpy : 0.00021600723266601562 nb_pixel_total : 3805 time to create 1 rle with old method : 0.004255056381225586 length of segment : 72 time for calcul the mask position with numpy : 0.0024671554565429688 nb_pixel_total : 93740 time to create 1 rle with old method : 0.10095930099487305 length of segment : 546 time for calcul the mask position with numpy : 0.0003044605255126953 nb_pixel_total : 3178 time to create 1 rle with old method : 0.003676176071166992 length of segment : 117 time for calcul the mask position with numpy : 0.0009386539459228516 nb_pixel_total : 35427 time to create 1 rle with old method : 0.03686046600341797 length of segment : 355 time for calcul the mask position with numpy : 0.023794174194335938 nb_pixel_total : 1133674 time to create 1 rle with new method : 0.2342231273651123 length of segment : 1543 time for calcul the mask position with numpy : 0.0013017654418945312 nb_pixel_total : 84875 time to create 1 rle with old method : 0.09799408912658691 length of segment : 430 time for calcul the mask position with numpy : 0.0005676746368408203 nb_pixel_total : 11951 time to create 1 rle with old method : 0.012766122817993164 length of segment : 145 time for calcul the mask position with numpy : 0.00025153160095214844 nb_pixel_total : 7471 time to create 1 rle with old method : 0.008015155792236328 length of segment : 110 time for calcul the mask position with numpy : 0.0003070831298828125 nb_pixel_total : 6901 time to create 1 rle with old method : 0.007409572601318359 length of segment : 136 time for calcul the mask position with numpy : 0.0002269744873046875 nb_pixel_total : 7265 time to create 1 rle with old method : 0.008048057556152344 length of segment : 100 time for calcul the mask position with numpy : 0.00029087066650390625 nb_pixel_total : 8264 time to create 1 rle with old method : 0.009209871292114258 length of segment : 105 time for calcul the mask position with numpy : 0.00020074844360351562 nb_pixel_total : 12633 time to create 1 rle with old method : 0.014461517333984375 length of segment : 147 time for calcul the mask position with numpy : 0.0001163482666015625 nb_pixel_total : 2439 time to create 1 rle with old method : 0.0031440258026123047 length of segment : 33 time for calcul the mask position with numpy : 0.00020933151245117188 nb_pixel_total : 5453 time to create 1 rle with old method : 0.006341457366943359 length of segment : 71 time for calcul the mask position with numpy : 0.0002779960632324219 nb_pixel_total : 14131 time to create 1 rle with old method : 0.015974760055541992 length of segment : 186 time spent for convertir_results : 2.7249646186828613 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 34 chid ids of type : 3594 Number RLEs to save : 9244 save missing photos in datou_result : time spend for datou_step_exec : 26.623011589050293 time spend to save output : 0.6111600399017334 total time spend for step 1 : 27.234171628952026 step2:crop_condition Sat Nov 15 09:50:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 17 ! batch 1 Loaded 34 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! About to upload 17 photos upload in portfolio : 3736932 init cache_photo without model_param we have 17 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1763196658_1007243 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516409_0.png', 0, 276, 204, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516417_0.png', 0, 177, 109, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516415_0.png', 0, 59, 80, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552965_29433d28677ecfde70f8738cb3e8561d_rle_crop_4034516419_0.png', 0, 68, 65, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552863_5334022d1787a5215ab664f1413a119a_rle_crop_4034516422_0.png', 0, 108, 62, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516424_0.png', 0, 91, 126, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516423_0.png', 0, 140, 92, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552752_a47a0a5c06ec816846120009dd7da808_rle_crop_4034516429_0.png', 0, 65, 116, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516434_0.png', 0, 107, 107, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516435_0.png', 0, 127, 117, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516437_0.png', 0, 127, 105, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516436_0.png', 0, 115, 93, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516433_0.png', 0, 163, 145, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516441_0.png', 0, 128, 167, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516438_0.png', 0, 123, 147, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516440_0.png', 0, 133, 67, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196661), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516439_0.png', 0, 126, 28, 0, 1763196661,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 17 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.713622808456421 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3736932 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1763196662_1007243 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196663), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464_rle_crop_4034516411_0.png', 0, 176, 101, 0, 1763196663,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196663), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516418_0.png', 0, 59, 44, 0, 1763196663,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196663), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d_rle_crop_4034516427_0.png', 0, 78, 72, 0, 1763196663,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.919475793838501 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3736932 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1763196677_1007243 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516408_0.png', 0, 91, 142, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464_rle_crop_4034516412_0.png', 0, 1101, 962, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552974_5e4edcb6563fe3a7b3438815f4caed97_rle_crop_4034516413_0.png', 0, 304, 533, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552971_a9a176c232e97952915f10f12af7e09e_rle_crop_4034516414_0.png', 0, 58, 67, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516416_0.png', 0, 336, 532, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905_rle_crop_4034516420_0.png', 0, 154, 153, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905_rle_crop_4034516421_0.png', 0, 339, 520, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516425_0.png', 0, 224, 129, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552800_3a27ee30c9eba2ee062b492856e8ba69_rle_crop_4034516426_0.png', 0, 1480, 992, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d_rle_crop_4034516428_0.png', 0, 284, 543, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516431_0.png', 0, 1399, 1003, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516432_0.png', 0, 318, 424, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196680), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516430_0.png', 0, 175, 343, 0, 1763196680,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 13 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.5456974506378174 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1763196681_1007243 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1763196682), 0.0, 0.0, 14, '', 0, 0, '1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516410_0.png', 0, 115, 90, 0, 1763196682,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.611609697341919 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1394552992, 1394552990, 1394552974, 1394552971, 1394552968, 1394552965, 1394552964, 1394552960, 1394552863, 1394552833, 1394552800, 1394552770, 1394552752, 1394552563, 1394552533, 1394552497, 1394552465] Looping around the photos to save general results len do output : 34 /1394559066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1394559271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552992', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552990', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552974', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552971', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552968', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552965', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552964', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552960', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552863', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552833', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552800', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552770', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552752', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552563', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552533', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552497', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552465', None, None, None, None, None, '4053083') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 119 time used for this insertion : 0.023592472076416016 save_final save missing photos in datou_result : time spend for datou_step_exec : 25.0233895778656 time spend to save output : 0.02524542808532715 total time spend for step 2 : 25.048635005950928 step3:rle_unique_nms_with_priority Sat Nov 15 09:51:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 34 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.3916044235229492 time for calcul the mask position with numpy : 0.06187152862548828 nb_pixel_total : 2021330 time to create 1 rle with new method : 0.29233622550964355 time for calcul the mask position with numpy : 0.006256103515625 nb_pixel_total : 6481 time to create 1 rle with old method : 0.007438182830810547 time for calcul the mask position with numpy : 0.006560087203979492 nb_pixel_total : 36487 time to create 1 rle with old method : 0.04015159606933594 time for calcul the mask position with numpy : 0.006525278091430664 nb_pixel_total : 9302 time to create 1 rle with old method : 0.010399580001831055 create new chi : 0.44107723236083984 time to delete rle : 0.025186538696289062 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1974 TO DO : save crop sub photo not yet done ! save time : 0.16149473190307617 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.10106873512268066 time for calcul the mask position with numpy : 0.037378787994384766 nb_pixel_total : 1347517 time to create 1 rle with new method : 0.2960526943206787 time for calcul the mask position with numpy : 0.011373758316040039 nb_pixel_total : 713681 time to create 1 rle with new method : 0.1438126564025879 time for calcul the mask position with numpy : 0.006345272064208984 nb_pixel_total : 12402 time to create 1 rle with old method : 0.013490915298461914 create new chi : 0.5155718326568604 time to delete rle : 0.0004937648773193359 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3278 TO DO : save crop sub photo not yet done ! save time : 0.25350427627563477 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.04303717613220215 time for calcul the mask position with numpy : 0.019158124923706055 nb_pixel_total : 1972610 time to create 1 rle with new method : 0.08859944343566895 time for calcul the mask position with numpy : 0.0074765682220458984 nb_pixel_total : 100990 time to create 1 rle with old method : 0.11234545707702637 create new chi : 0.2363739013671875 time to delete rle : 0.0003044605255126953 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2148 TO DO : save crop sub photo not yet done ! save time : 0.1811368465423584 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.13569211959838867 time for calcul the mask position with numpy : 0.1202549934387207 nb_pixel_total : 2070514 time to create 1 rle with new method : 0.09103178977966309 time for calcul the mask position with numpy : 0.00594782829284668 nb_pixel_total : 3086 time to create 1 rle with old method : 0.0033931732177734375 create new chi : 0.23059749603271484 time to delete rle : 0.00023794174194335938 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1218 TO DO : save crop sub photo not yet done ! save time : 0.11505413055419922 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.2073507308959961 time for calcul the mask position with numpy : 0.2435007095336914 nb_pixel_total : 1945226 time to create 1 rle with new method : 0.22248601913452148 time for calcul the mask position with numpy : 0.006856441497802734 nb_pixel_total : 1827 time to create 1 rle with old method : 0.0020818710327148438 time for calcul the mask position with numpy : 0.006940603256225586 nb_pixel_total : 13186 time to create 1 rle with old method : 0.014904260635375977 time for calcul the mask position with numpy : 0.007591962814331055 nb_pixel_total : 110030 time to create 1 rle with old method : 0.20113897323608398 time for calcul the mask position with numpy : 0.007417201995849609 nb_pixel_total : 3331 time to create 1 rle with old method : 0.004312038421630859 create new chi : 0.7296121120452881 time to delete rle : 0.0003714561462402344 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2708 TO DO : save crop sub photo not yet done ! save time : 0.21386218070983887 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.035923004150390625 time for calcul the mask position with numpy : 0.020821332931518555 nb_pixel_total : 2070967 time to create 1 rle with new method : 0.053603410720825195 time for calcul the mask position with numpy : 0.0065076351165771484 nb_pixel_total : 2633 time to create 1 rle with old method : 0.003060579299926758 create new chi : 0.08420825004577637 time to delete rle : 0.0002181529998779297 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 1212 TO DO : save crop sub photo not yet done ! save time : 0.11113667488098145 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.044446706771850586 time for calcul the mask position with numpy : 0.020294189453125 nb_pixel_total : 1950586 time to create 1 rle with new method : 0.05819344520568848 time for calcul the mask position with numpy : 0.006663799285888672 nb_pixel_total : 105094 time to create 1 rle with old method : 0.12372422218322754 time for calcul the mask position with numpy : 0.00664520263671875 nb_pixel_total : 17920 time to create 1 rle with old method : 0.02278447151184082 create new chi : 0.24556922912597656 time to delete rle : 0.000438690185546875 batch 1 Loaded 5 chid ids of type : 3594 +++Number RLEs to save : 2460 TO DO : save crop sub photo not yet done ! save time : 0.1826171875 No data in photo_id : 1394552960 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03393888473510742 time for calcul the mask position with numpy : 0.020006179809570312 nb_pixel_total : 2068132 time to create 1 rle with new method : 0.25865840911865234 time for calcul the mask position with numpy : 0.0068514347076416016 nb_pixel_total : 5468 time to create 1 rle with old method : 0.006422281265258789 create new chi : 0.3032658100128174 time to delete rle : 0.00028777122497558594 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1212 TO DO : save crop sub photo not yet done ! save time : 0.12029147148132324 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.12053227424621582 time for calcul the mask position with numpy : 0.11424446105957031 nb_pixel_total : 2035879 time to create 1 rle with new method : 0.2383103370666504 time for calcul the mask position with numpy : 0.007040262222290039 nb_pixel_total : 20355 time to create 1 rle with old method : 0.025334835052490234 time for calcul the mask position with numpy : 0.0069348812103271484 nb_pixel_total : 7476 time to create 1 rle with old method : 0.009135246276855469 time for calcul the mask position with numpy : 0.006775856018066406 nb_pixel_total : 9890 time to create 1 rle with old method : 0.01156926155090332 create new chi : 0.43159985542297363 time to delete rle : 0.0004024505615234375 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1776 TO DO : save crop sub photo not yet done ! save time : 0.1660783290863037 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03683876991271973 time for calcul the mask position with numpy : 0.011870861053466797 nb_pixel_total : 839762 time to create 1 rle with new method : 0.07051491737365723 time for calcul the mask position with numpy : 0.014293432235717773 nb_pixel_total : 1233838 time to create 1 rle with new method : 0.027836084365844727 create new chi : 0.12492680549621582 time to delete rle : 0.0003123283386230469 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 3110 TO DO : save crop sub photo not yet done ! save time : 0.23907041549682617 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.25905370712280273 time for calcul the mask position with numpy : 0.1264331340789795 nb_pixel_total : 1976055 time to create 1 rle with new method : 0.1880333423614502 time for calcul the mask position with numpy : 0.0066623687744140625 nb_pixel_total : 93740 time to create 1 rle with old method : 0.10102510452270508 time for calcul the mask position with numpy : 0.0062961578369140625 nb_pixel_total : 3805 time to create 1 rle with old method : 0.004250526428222656 create new chi : 0.4432394504547119 time to delete rle : 0.0005297660827636719 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2316 TO DO : save crop sub photo not yet done ! save time : 0.18878173828125 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03415346145629883 time for calcul the mask position with numpy : 0.04076409339904785 nb_pixel_total : 2070422 time to create 1 rle with new method : 0.15642213821411133 time for calcul the mask position with numpy : 0.005955219268798828 nb_pixel_total : 3178 time to create 1 rle with old method : 0.003441333770751953 create new chi : 0.21561408042907715 time to delete rle : 0.00022912025451660156 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1314 TO DO : save crop sub photo not yet done ! save time : 0.1306440830230713 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 0.08796525001525879 time for calcul the mask position with numpy : 0.1174020767211914 nb_pixel_total : 824231 time to create 1 rle with new method : 0.08771395683288574 time for calcul the mask position with numpy : 0.006967782974243164 nb_pixel_total : 84875 time to create 1 rle with old method : 0.09244012832641602 time for calcul the mask position with numpy : 0.08867740631103516 nb_pixel_total : 1129067 time to create 1 rle with new method : 0.07166457176208496 time for calcul the mask position with numpy : 0.0059010982513427734 nb_pixel_total : 35427 time to create 1 rle with old method : 0.03789210319519043 create new chi : 0.5257360935211182 time to delete rle : 0.000667572021484375 batch 1 Loaded 7 chid ids of type : 3594 +++++++Number RLEs to save : 5614 TO DO : save crop sub photo not yet done ! save time : 0.36501049995422363 nb_obj : 5 nb_hashtags : 1 time to prepare the origin masks : 0.20821452140808105 time for calcul the mask position with numpy : 0.09448862075805664 nb_pixel_total : 2031748 time to create 1 rle with new method : 0.07829618453979492 time for calcul the mask position with numpy : 0.005831003189086914 nb_pixel_total : 8264 time to create 1 rle with old method : 0.008683204650878906 time for calcul the mask position with numpy : 0.005651712417602539 nb_pixel_total : 7265 time to create 1 rle with old method : 0.007649660110473633 time for calcul the mask position with numpy : 0.00566554069519043 nb_pixel_total : 6901 time to create 1 rle with old method : 0.007188081741333008 time for calcul the mask position with numpy : 0.005618095397949219 nb_pixel_total : 7471 time to create 1 rle with old method : 0.00771021842956543 time for calcul the mask position with numpy : 0.005643129348754883 nb_pixel_total : 11951 time to create 1 rle with old method : 0.0124359130859375 create new chi : 0.2553706169128418 time to delete rle : 0.00037217140197753906 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 2272 TO DO : save crop sub photo not yet done ! save time : 0.17282986640930176 nb_obj : 4 nb_hashtags : 1 time to prepare the origin masks : 0.06109476089477539 time for calcul the mask position with numpy : 0.12539005279541016 nb_pixel_total : 2038944 time to create 1 rle with new method : 0.0751798152923584 time for calcul the mask position with numpy : 0.006043910980224609 nb_pixel_total : 14131 time to create 1 rle with old method : 0.015306949615478516 time for calcul the mask position with numpy : 0.005932331085205078 nb_pixel_total : 5453 time to create 1 rle with old method : 0.005736351013183594 time for calcul the mask position with numpy : 0.005831480026245117 nb_pixel_total : 2439 time to create 1 rle with old method : 0.0027282238006591797 time for calcul the mask position with numpy : 0.0060329437255859375 nb_pixel_total : 12633 time to create 1 rle with old method : 0.014617919921875 create new chi : 0.27200913429260254 time to delete rle : 0.0004489421844482422 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 1954 TO DO : save crop sub photo not yet done ! save time : 0.1654496192932129 No data in photo_id : 1394552465 map_output_result : {1394552992: (0.0, 'Should be the crop_list due to order', 0), 1394552990: (0.0, 'Should be the crop_list due to order', 0), 1394552974: (0.0, 'Should be the crop_list due to order', 0), 1394552971: (0.0, 'Should be the crop_list due to order', 0), 1394552968: (0.0, 'Should be the crop_list due to order', 0), 1394552965: (0.0, 'Should be the crop_list due to order', 0), 1394552964: (0.0, 'Should be the crop_list due to order', 0), 1394552960: (0.0, 'Should be the crop_list due to order', 0.0), 1394552863: (0.0, 'Should be the crop_list due to order', 0), 1394552833: (0.0, 'Should be the crop_list due to order', 0), 1394552800: (0.0, 'Should be the crop_list due to order', 0), 1394552770: (0.0, 'Should be the crop_list due to order', 0), 1394552752: (0.0, 'Should be the crop_list due to order', 0), 1394552563: (0.0, 'Should be the crop_list due to order', 0), 1394552533: (0.0, 'Should be the crop_list due to order', 0), 1394552497: (0.0, 'Should be the crop_list due to order', 0), 1394552465: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1394552992, 1394552990, 1394552974, 1394552971, 1394552968, 1394552965, 1394552964, 1394552960, 1394552863, 1394552833, 1394552800, 1394552770, 1394552752, 1394552563, 1394552533, 1394552497, 1394552465] Looping around the photos to save general results len do output : 17 /1394552992.Didn't retrieve data . /1394552990.Didn't retrieve data . /1394552974.Didn't retrieve data . /1394552971.Didn't retrieve data . /1394552968.Didn't retrieve data . /1394552965.Didn't retrieve data . /1394552964.Didn't retrieve data . /1394552960.Didn't retrieve data . /1394552863.Didn't retrieve data . /1394552833.Didn't retrieve data . /1394552800.Didn't retrieve data . /1394552770.Didn't retrieve data . /1394552752.Didn't retrieve data . /1394552563.Didn't retrieve data . /1394552533.Didn't retrieve data . /1394552497.Didn't retrieve data . /1394552465.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552992', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552990', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552974', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552971', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552968', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552965', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552964', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552960', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552863', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552833', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552800', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552770', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552752', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552563', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552533', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552497', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552465', None, None, None, None, None, '4053083') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 51 time used for this insertion : 0.016492605209350586 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.171432971954346 time spend to save output : 0.017049312591552734 total time spend for step 3 : 10.188482284545898 step4:ventilate_hashtags_in_portfolio Sat Nov 15 09:51:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 28565337 get user id for portfolio 28565337 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28565337 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','background','mal_croppe','pet_clair','autre','papier','pehd','metal','environnement','carton','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28565337 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','background','mal_croppe','pet_clair','autre','papier','pehd','metal','environnement','carton','flou')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28565337 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('pet_fonce','background','mal_croppe','pet_clair','autre','papier','pehd','metal','environnement','carton','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/28565448,28565449,28565450,28565451,28565452,28565453,28565454,28565455,28565456,28565457,28565458?tags=pet_fonce,background,mal_croppe,pet_clair,autre,papier,pehd,metal,environnement,carton,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1394552992, 1394552990, 1394552974, 1394552971, 1394552968, 1394552965, 1394552964, 1394552960, 1394552863, 1394552833, 1394552800, 1394552770, 1394552752, 1394552563, 1394552533, 1394552497, 1394552465] Looping around the photos to save general results len do output : 1 /28565337. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552992', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552990', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552974', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552971', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552968', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552965', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552964', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552960', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552863', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552833', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552800', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552770', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552752', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552563', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552533', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552497', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552465', None, None, None, None, None, '4053083') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.017385482788085938 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7038702964782715 time spend to save output : 0.01768040657043457 total time spend for step 4 : 0.721550703048706 step5:final Sat Nov 15 09:51:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1394552992: ('0.10893464619644158',), 1394552990: ('0.10893464619644158',), 1394552974: ('0.10893464619644158',), 1394552971: ('0.10893464619644158',), 1394552968: ('0.10893464619644158',), 1394552965: ('0.10893464619644158',), 1394552964: ('0.10893464619644158',), 1394552960: ('0.10893464619644158',), 1394552863: ('0.10893464619644158',), 1394552833: ('0.10893464619644158',), 1394552800: ('0.10893464619644158',), 1394552770: ('0.10893464619644158',), 1394552752: ('0.10893464619644158',), 1394552563: ('0.10893464619644158',), 1394552533: ('0.10893464619644158',), 1394552497: ('0.10893464619644158',), 1394552465: ('0.10893464619644158',)} new output for save of step final : {1394552992: ('0.10893464619644158',), 1394552990: ('0.10893464619644158',), 1394552974: ('0.10893464619644158',), 1394552971: ('0.10893464619644158',), 1394552968: ('0.10893464619644158',), 1394552965: ('0.10893464619644158',), 1394552964: ('0.10893464619644158',), 1394552960: ('0.10893464619644158',), 1394552863: ('0.10893464619644158',), 1394552833: ('0.10893464619644158',), 1394552800: ('0.10893464619644158',), 1394552770: ('0.10893464619644158',), 1394552752: ('0.10893464619644158',), 1394552563: ('0.10893464619644158',), 1394552533: ('0.10893464619644158',), 1394552497: ('0.10893464619644158',), 1394552465: ('0.10893464619644158',)} [1394552992, 1394552990, 1394552974, 1394552971, 1394552968, 1394552965, 1394552964, 1394552960, 1394552863, 1394552833, 1394552800, 1394552770, 1394552752, 1394552563, 1394552533, 1394552497, 1394552465] Looping around the photos to save general results len do output : 17 /1394552992.Didn't retrieve data . /1394552990.Didn't retrieve data . /1394552974.Didn't retrieve data . /1394552971.Didn't retrieve data . /1394552968.Didn't retrieve data . /1394552965.Didn't retrieve data . /1394552964.Didn't retrieve data . /1394552960.Didn't retrieve data . /1394552863.Didn't retrieve data . /1394552833.Didn't retrieve data . /1394552800.Didn't retrieve data . /1394552770.Didn't retrieve data . /1394552752.Didn't retrieve data . /1394552563.Didn't retrieve data . /1394552533.Didn't retrieve data . /1394552497.Didn't retrieve data . /1394552465.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552992', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552990', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552974', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552971', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552968', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552965', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552964', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552960', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552863', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552833', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552800', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552770', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552752', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552563', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552533', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552497', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552465', None, None, None, None, None, '4053083') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 51 time used for this insertion : 0.01695728302001953 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12149691581726074 time spend to save output : 0.017575740814208984 total time spend for step 5 : 0.13907265663146973 step6:blur_detection Sat Nov 15 09:51:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008.jpg resize: (1080, 1920) 1394552992 0.013325524226165427 treat image : temp/1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464.jpg resize: (1080, 1920) 1394552990 2.0610430072573562 treat image : temp/1763196627_1007243_1394552974_5e4edcb6563fe3a7b3438815f4caed97.jpg resize: (1080, 1920) 1394552974 -3.9569001599959224 treat image : temp/1763196627_1007243_1394552971_a9a176c232e97952915f10f12af7e09e.jpg resize: (1080, 1920) 1394552971 -0.05573519043146407 treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef.jpg resize: (1080, 1920) 1394552968 -3.975534522209864 treat image : temp/1763196627_1007243_1394552965_29433d28677ecfde70f8738cb3e8561d.jpg resize: (1080, 1920) 1394552965 0.0737220689691678 treat image : temp/1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905.jpg resize: (1080, 1920) 1394552964 -3.1113576395170885 treat image : temp/1763196627_1007243_1394552960_969b533168c6af23f9729eeefc9df57a.jpg resize: (1080, 1920) 1394552960 -3.3957622265378937 treat image : temp/1763196627_1007243_1394552863_5334022d1787a5215ab664f1413a119a.jpg resize: (1080, 1920) 1394552863 -0.28927985333744183 treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24.jpg resize: (1080, 1920) 1394552833 -4.09566810493006 treat image : temp/1763196627_1007243_1394552800_3a27ee30c9eba2ee062b492856e8ba69.jpg resize: (1080, 1920) 1394552800 1.7168877707898422 treat image : temp/1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d.jpg resize: (1080, 1920) 1394552770 -4.605217894473731 treat image : temp/1763196627_1007243_1394552752_a47a0a5c06ec816846120009dd7da808.jpg resize: (1080, 1920) 1394552752 -1.7631177511918898 treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50.jpg resize: (1080, 1920) 1394552563 0.12053219523815872 treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24.jpg resize: (1080, 1920) 1394552533 1.8767598719585936 treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3.jpg resize: (1080, 1920) 1394552497 -0.28954511423756996 treat image : temp/1763196627_1007243_1394552465_54ec5463546d9b16545ad497530cc060.jpg resize: (1080, 1920) 1394552465 0.4540354890863428 treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516409_0.png resize: (204, 276) 1394559066 -1.6726194584969654 treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516417_0.png resize: (109, 177) 1394559067 -4.289071674029468 treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516415_0.png resize: (80, 59) 1394559068 -1.7834353421711453 treat image : temp/1763196627_1007243_1394552965_29433d28677ecfde70f8738cb3e8561d_rle_crop_4034516419_0.png resize: (65, 68) 1394559069 -0.5703365886395385 treat image : temp/1763196627_1007243_1394552863_5334022d1787a5215ab664f1413a119a_rle_crop_4034516422_0.png resize: (62, 108) 1394559070 1.2913675976013683 treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516424_0.png resize: (126, 91) 1394559071 -0.8210308908642074 treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516423_0.png resize: (92, 140) 1394559072 -1.3713171275039093 treat image : temp/1763196627_1007243_1394552752_a47a0a5c06ec816846120009dd7da808_rle_crop_4034516429_0.png resize: (116, 65) 1394559073 -2.2898547633181887 treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516434_0.png resize: (107, 107) 1394559074 1.3106622434077446 treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516435_0.png resize: (117, 127) 1394559075 -1.4717694751169605 treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516437_0.png resize: (105, 127) 1394559076 -1.160459804969517 treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516436_0.png resize: (93, 115) 1394559077 -1.671486083062162 treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516433_0.png resize: (145, 163) 1394559078 -1.660268110280159 treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516441_0.png resize: (167, 128) 1394559079 -1.7279315258678691 treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516438_0.png resize: (147, 123) 1394559080 -0.0647115967392412 treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516440_0.png resize: (67, 133) 1394559081 -3.145776473773086 treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516439_0.png resize: (28, 126) 1394559082 -0.9123422698015832 treat image : temp/1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464_rle_crop_4034516411_0.png resize: (101, 176) 1394559083 -1.6529504183303279 treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516418_0.png resize: (44, 59) 1394559084 -1.113503328849315 treat image : temp/1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d_rle_crop_4034516427_0.png resize: (72, 78) 1394559085 -1.897161859777985 treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516408_0.png resize: (142, 91) 1394559245 -1.8734943945247426 treat image : temp/1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464_rle_crop_4034516412_0.png resize: (962, 1101) 1394559246 0.028872012721966805 treat image : temp/1763196627_1007243_1394552974_5e4edcb6563fe3a7b3438815f4caed97_rle_crop_4034516413_0.png resize: (533, 304) 1394559248 0.19935886515392834 treat image : temp/1763196627_1007243_1394552971_a9a176c232e97952915f10f12af7e09e_rle_crop_4034516414_0.png resize: (67, 58) 1394559249 0.1960278903645535 treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516416_0.png resize: (532, 336) 1394559250 0.1726388399175247 treat image : temp/1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905_rle_crop_4034516420_0.png resize: (153, 154) 1394559252 -2.729999304821861 treat image : temp/1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905_rle_crop_4034516421_0.png resize: (520, 339) 1394559253 -0.4495233606463083 treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516425_0.png resize: (129, 224) 1394559254 -2.44096931296845 treat image : temp/1763196627_1007243_1394552800_3a27ee30c9eba2ee062b492856e8ba69_rle_crop_4034516426_0.png resize: (992, 1480) 1394559256 0.7761340665137685 treat image : temp/1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d_rle_crop_4034516428_0.png resize: (543, 284) 1394559257 -0.2376973202125332 treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516431_0.png resize: (1003, 1399) 1394559258 -0.999092173896101 treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516432_0.png resize: (424, 318) 1394559260 -0.27254051050801986 treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516430_0.png resize: (343, 175) 1394559261 -0.765959079120761 treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516410_0.png resize: (90, 115) 1394559271 -1.8903498616802659 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 51 time used for this insertion : 0.017001867294311523 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 51 time used for this insertion : 0.01697683334350586 save missing photos in datou_result : time spend for datou_step_exec : 14.5688316822052 time spend to save output : 0.03878498077392578 total time spend for step 6 : 14.607616662979126 step7:brightness Sat Nov 15 09:51:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008.jpg treat image : temp/1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464.jpg treat image : temp/1763196627_1007243_1394552974_5e4edcb6563fe3a7b3438815f4caed97.jpg treat image : temp/1763196627_1007243_1394552971_a9a176c232e97952915f10f12af7e09e.jpg treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef.jpg treat image : temp/1763196627_1007243_1394552965_29433d28677ecfde70f8738cb3e8561d.jpg treat image : temp/1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905.jpg treat image : temp/1763196627_1007243_1394552960_969b533168c6af23f9729eeefc9df57a.jpg treat image : temp/1763196627_1007243_1394552863_5334022d1787a5215ab664f1413a119a.jpg treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24.jpg treat image : temp/1763196627_1007243_1394552800_3a27ee30c9eba2ee062b492856e8ba69.jpg treat image : temp/1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d.jpg treat image : temp/1763196627_1007243_1394552752_a47a0a5c06ec816846120009dd7da808.jpg treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50.jpg treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24.jpg treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3.jpg treat image : temp/1763196627_1007243_1394552465_54ec5463546d9b16545ad497530cc060.jpg treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516409_0.png treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516417_0.png treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516415_0.png treat image : temp/1763196627_1007243_1394552965_29433d28677ecfde70f8738cb3e8561d_rle_crop_4034516419_0.png treat image : temp/1763196627_1007243_1394552863_5334022d1787a5215ab664f1413a119a_rle_crop_4034516422_0.png treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516424_0.png treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516423_0.png treat image : temp/1763196627_1007243_1394552752_a47a0a5c06ec816846120009dd7da808_rle_crop_4034516429_0.png treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516434_0.png treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516435_0.png treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516437_0.png treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516436_0.png treat image : temp/1763196627_1007243_1394552533_a36c4e636b5b2d0dc5eb81fbfd096d24_rle_crop_4034516433_0.png treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516441_0.png treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516438_0.png treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516440_0.png treat image : temp/1763196627_1007243_1394552497_fee45b3a1e2a9a9aec0fa91ca6d1b0c3_rle_crop_4034516439_0.png treat image : temp/1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464_rle_crop_4034516411_0.png treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516418_0.png treat image : temp/1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d_rle_crop_4034516427_0.png treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516408_0.png treat image : temp/1763196627_1007243_1394552990_ea08bf5747bfd098edbf2a3053f63464_rle_crop_4034516412_0.png treat image : temp/1763196627_1007243_1394552974_5e4edcb6563fe3a7b3438815f4caed97_rle_crop_4034516413_0.png treat image : temp/1763196627_1007243_1394552971_a9a176c232e97952915f10f12af7e09e_rle_crop_4034516414_0.png treat image : temp/1763196627_1007243_1394552968_5f7d3ef22a0b34e899cf677b2b68c0ef_rle_crop_4034516416_0.png treat image : temp/1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905_rle_crop_4034516420_0.png treat image : temp/1763196627_1007243_1394552964_8f2ed3c69a1116e5ea5d6de69b2c0905_rle_crop_4034516421_0.png treat image : temp/1763196627_1007243_1394552833_982a4ede63f9b92ff33d8050a5209e24_rle_crop_4034516425_0.png treat image : temp/1763196627_1007243_1394552800_3a27ee30c9eba2ee062b492856e8ba69_rle_crop_4034516426_0.png treat image : temp/1763196627_1007243_1394552770_d4d193be98683ff3a0e65987e894e45d_rle_crop_4034516428_0.png treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516431_0.png treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516432_0.png treat image : temp/1763196627_1007243_1394552563_c0cdfc51080792bf58bc27046b585d50_rle_crop_4034516430_0.png treat image : temp/1763196627_1007243_1394552992_f759d9511387c9cef158edebaaa1c008_rle_crop_4034516410_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 51 time used for this insertion : 0.01504206657409668 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 51 time used for this insertion : 0.014606952667236328 save missing photos in datou_result : time spend for datou_step_exec : 4.2116379737854 time spend to save output : 0.03477740287780762 total time spend for step 7 : 4.246415376663208 step8:velours_tree Sat Nov 15 09:51:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.08105993270874023 time spend to save output : 3.695487976074219e-05 total time spend for step 8 : 0.08109688758850098 step9:send_mail_cod Sat Nov 15 09:51:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P28565337_15-11-2025_09_51_52.pdf 28565448 imagette285654481763196712 28565449 imagette285654491763196712 28565450 imagette285654501763196712 28565451 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette285654511763196712 28565452 change filename to text .imagette285654521763196713 28565453 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette285654531763196713 28565454 imagette285654541763196714 28565455 imagette285654551763196714 28565457 change filename to text .change filename to text .change filename to text .imagette285654571763196714 28565458 imagette285654581763196714 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=28565337 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/28565448,28565449,28565450,28565451,28565452,28565453,28565454,28565455,28565456,28565457,28565458?tags=pet_fonce,background,mal_croppe,pet_clair,autre,papier,pehd,metal,environnement,carton,flou args[1394552992] : ((1394552992, 0.013325524226165427, 492688767), (1394552992, 0.3745051061175087, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552990] : ((1394552990, 2.0610430072573562, 492688767), (1394552990, 0.25810295013300544, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552974] : ((1394552974, -3.9569001599959224, 492609224), (1394552974, 0.3109118075712556, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552971] : ((1394552971, -0.05573519043146407, 492688767), (1394552971, 0.5641708324826537, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552968] : ((1394552968, -3.975534522209864, 492609224), (1394552968, 0.3578988799960053, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552965] : ((1394552965, 0.0737220689691678, 492688767), (1394552965, 0.29840736374625076, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552964] : ((1394552964, -3.1113576395170885, 492609224), (1394552964, -0.14157660690945148, 496442774), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552960] : ((1394552960, -3.3957622265378937, 492609224), (1394552960, 0.2966903107699386, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552863] : ((1394552863, -0.28927985333744183, 492688767), (1394552863, 0.8809015739722614, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552833] : ((1394552833, -4.09566810493006, 492609224), (1394552833, 0.5342855072472708, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552800] : ((1394552800, 1.7168877707898422, 492688767), (1394552800, 0.6880454202847563, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552770] : ((1394552770, -4.605217894473731, 492609224), (1394552770, 0.22027045324995695, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552752] : ((1394552752, -1.7631177511918898, 492688767), (1394552752, 1.396429375266433, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552563] : ((1394552563, 0.12053219523815872, 492688767), (1394552563, 0.6555960437168413, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552533] : ((1394552533, 1.8767598719585936, 492688767), (1394552533, 0.7550778118115131, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552497] : ((1394552497, -0.28954511423756996, 492688767), (1394552497, 0.13146084355510584, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com args[1394552465] : ((1394552465, 0.4540354890863428, 492688767), (1394552465, 0.9073201675659056, 2107752395), '0.10893464619644158') We are sending mail with results at report@fotonower.com refus_total : 0.10893464619644158 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos_view ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=28565337 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28565337_15-11-2025_09_51_52.pdf results_Auto_P28565337_15-11-2025_09_51_52.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28565337_15-11-2025_09_51_52.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','28565337','results_Auto_P28565337_15-11-2025_09_51_52.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28565337_15-11-2025_09_51_52.pdf','pdf','','0.56','0.10893464619644158') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/28565337

https://www.fotonower.com/image?json=false&list_photos_id=1394552992
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552990
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.0610430072573562)
https://www.fotonower.com/image?json=false&list_photos_id=1394552974
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552971
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552968
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552965
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552964
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552960
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552863
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552833
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552800
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.7168877707898422)
https://www.fotonower.com/image?json=false&list_photos_id=1394552770
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552752
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552563
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552533
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.8767598719585936)
https://www.fotonower.com/image?json=false&list_photos_id=1394552497
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1394552465
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 10.89%
Veuillez trouver les photos des contaminants.

exemples de contaminants: pet_clair: https://www.fotonower.com/view/28565451?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/28565452?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/28565453?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/28565457?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28565337_15-11-2025_09_51_52.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/28565448,28565449,28565450,28565451,28565452,28565453,28565454,28565455,28565456,28565457,28565458?tags=pet_fonce,background,mal_croppe,pet_clair,autre,papier,pehd,metal,environnement,carton,flou.


L'équipe Fotonower 202 b'' Date: Sat, 15 Nov 2025 08:51:57 GMT Content-Length: 0 Connection: close Server: nginx X-Message-Id: NrLedePHRzi5U3YKIN4FvQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1394552992, 1394552990, 1394552974, 1394552971, 1394552968, 1394552965, 1394552964, 1394552960, 1394552863, 1394552833, 1394552800, 1394552770, 1394552752, 1394552563, 1394552533, 1394552497, 1394552465] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552992', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552990', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552974', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552971', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552968', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552965', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552964', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552960', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552863', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552833', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552800', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552770', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552752', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552563', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552533', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552497', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552465', None, None, None, None, None, '4053083') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 17 time used for this insertion : 0.017223119735717773 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.011815786361694 time spend to save output : 0.017473220825195312 total time spend for step 9 : 5.02928900718689 step10:split_time_score Sat Nov 15 09:51:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('09', 17),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 15112025 28565337 Nombre de photos uploadées : 17 / 23040 (0%) 15112025 28565337 Nombre de photos taguées (types de déchets): 0 / 17 (0%) 15112025 28565337 Nombre de photos taguées (volume) : 0 / 17 (0%) elapsed_time : load_data_split_time_score 3.337860107421875e-06 elapsed_time : order_list_meta_photo_and_scores 8.106231689453125e-06 ????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0007035732269287109 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21622180938720703 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.08777456275720165 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28563679_15-11-2025_08_41_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28563679 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28563679 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28565334 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28565335 order by id desc limit 1 Qualite : 0.10893464619644158 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P28565337_15-11-2025_09_51_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 28565337 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=28565337 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'15112025': {'nb_upload': 17, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1394552992, 1394552990, 1394552974, 1394552971, 1394552968, 1394552965, 1394552964, 1394552960, 1394552863, 1394552833, 1394552800, 1394552770, 1394552752, 1394552563, 1394552533, 1394552497, 1394552465] Looping around the photos to save general results len do output : 1 /28565337Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552992', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552990', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552974', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552971', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552968', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552965', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552964', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552960', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552863', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552833', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552800', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552770', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552752', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552563', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552533', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552497', None, None, None, None, None, '4053083') ('3318', None, None, None, None, None, None, None, '4053083') ('3318', '28565337', '1394552465', None, None, None, None, None, '4053083') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.01868724822998047 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.9758074283599854 time spend to save output : 0.01895880699157715 total time spend for step 10 : 2.9947662353515625 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 17 set_done_treatment 56.73user 21.82system 1:35.91elapsed 81%CPU (0avgtext+0avgdata 2893032maxresident)k 495456inputs+39888outputs (151major+1775764minor)pagefaults 0swaps