python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3825190 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3433890'] with mtr_portfolio_ids : ['25638289'] and first list_photo_ids : [] new path : /proc/3825190/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 19 ; length of list_pids : 19 ; length of list_args : 19 time to download the photos : 2.304720640182495 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Mon Aug 4 13:10:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10957 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-04 13:10:33.227685: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-04 13:10:33.255331: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493035000 Hz 2025-08-04 13:10:33.257143: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fec40000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-04 13:10:33.257186: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-04 13:10:33.260738: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-04 13:10:33.376231: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x137a2590 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-04 13:10:33.376272: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-04 13:10:33.377599: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-04 13:10:33.377934: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-04 13:10:33.380591: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-04 13:10:33.383002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-04 13:10:33.383500: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-04 13:10:33.385893: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-04 13:10:33.387073: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-04 13:10:33.391758: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-04 13:10:33.393254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-04 13:10:33.393313: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-04 13:10:33.394052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-04 13:10:33.394067: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-04 13:10:33.394075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-04 13:10:33.395501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10011 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-04 13:10:33.645435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-04 13:10:33.645526: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-04 13:10:33.645553: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-04 13:10:33.645578: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-04 13:10:33.645602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-04 13:10:33.645626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-04 13:10:33.645649: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-04 13:10:33.645673: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-04 13:10:33.647807: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-04 13:10:33.649139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-04 13:10:33.649168: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-04 13:10:33.649183: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-04 13:10:33.649197: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-04 13:10:33.649210: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-04 13:10:33.649224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-04 13:10:33.649238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-04 13:10:33.649251: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-04 13:10:33.650540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-04 13:10:33.650566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-04 13:10:33.650574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-04 13:10:33.650582: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-04 13:10:33.651938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10011 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-04 13:10:42.177358: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-04 13:10:42.355091: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 19 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 36.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 19.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 42.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 1 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 36.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 1 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 12.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 39.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 44.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.20000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 Detection mask done ! Trying to reset tf kernel 3825783 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 737 tf kernel not reseted sub process len(results) : 19 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 19 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5984 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.000583648681640625 nb_pixel_total : 22467 time to create 1 rle with old method : 0.025542736053466797 length of segment : 289 time for calcul the mask position with numpy : 0.0001995563507080078 nb_pixel_total : 7564 time to create 1 rle with old method : 0.00897669792175293 length of segment : 70 time for calcul the mask position with numpy : 0.002050638198852539 nb_pixel_total : 111108 time to create 1 rle with old method : 0.12373018264770508 length of segment : 522 time for calcul the mask position with numpy : 0.00018286705017089844 nb_pixel_total : 5164 time to create 1 rle with old method : 0.005948066711425781 length of segment : 128 time for calcul the mask position with numpy : 0.00014662742614746094 nb_pixel_total : 6747 time to create 1 rle with old method : 0.008104801177978516 length of segment : 80 time for calcul the mask position with numpy : 0.0002949237823486328 nb_pixel_total : 18351 time to create 1 rle with old method : 0.02132868766784668 length of segment : 166 time for calcul the mask position with numpy : 0.0007586479187011719 nb_pixel_total : 44652 time to create 1 rle with old method : 0.05090141296386719 length of segment : 282 time for calcul the mask position with numpy : 0.00017452239990234375 nb_pixel_total : 6459 time to create 1 rle with old method : 0.007689476013183594 length of segment : 86 time for calcul the mask position with numpy : 0.0004246234893798828 nb_pixel_total : 23598 time to create 1 rle with old method : 0.027253150939941406 length of segment : 293 time for calcul the mask position with numpy : 0.0001614093780517578 nb_pixel_total : 7224 time to create 1 rle with old method : 0.008669137954711914 length of segment : 63 time for calcul the mask position with numpy : 0.0001544952392578125 nb_pixel_total : 6252 time to create 1 rle with old method : 0.007516145706176758 length of segment : 93 time for calcul the mask position with numpy : 0.00024199485778808594 nb_pixel_total : 11741 time to create 1 rle with old method : 0.014138936996459961 length of segment : 144 time for calcul the mask position with numpy : 0.00024700164794921875 nb_pixel_total : 11252 time to create 1 rle with old method : 0.013225555419921875 length of segment : 122 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 5565 time to create 1 rle with old method : 0.0067138671875 length of segment : 71 time for calcul the mask position with numpy : 0.0018088817596435547 nb_pixel_total : 111262 time to create 1 rle with old method : 0.1257317066192627 length of segment : 603 time for calcul the mask position with numpy : 0.0005598068237304688 nb_pixel_total : 23873 time to create 1 rle with old method : 0.0269315242767334 length of segment : 239 time for calcul the mask position with numpy : 0.00048041343688964844 nb_pixel_total : 17306 time to create 1 rle with old method : 0.020313739776611328 length of segment : 271 time for calcul the mask position with numpy : 0.0016527175903320312 nb_pixel_total : 101967 time to create 1 rle with old method : 0.11305809020996094 length of segment : 555 time for calcul the mask position with numpy : 0.0001437664031982422 nb_pixel_total : 6319 time to create 1 rle with old method : 0.007106304168701172 length of segment : 70 time for calcul the mask position with numpy : 0.00011777877807617188 nb_pixel_total : 5028 time to create 1 rle with old method : 0.005706310272216797 length of segment : 83 time for calcul the mask position with numpy : 0.00013756752014160156 nb_pixel_total : 6047 time to create 1 rle with old method : 0.006846427917480469 length of segment : 135 time for calcul the mask position with numpy : 0.0015788078308105469 nb_pixel_total : 117125 time to create 1 rle with old method : 0.12334465980529785 length of segment : 535 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 4708 time to create 1 rle with old method : 0.005152702331542969 length of segment : 95 time for calcul the mask position with numpy : 0.00011706352233886719 nb_pixel_total : 5677 time to create 1 rle with old method : 0.0065250396728515625 length of segment : 51 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 3197 time to create 1 rle with old method : 0.003608226776123047 length of segment : 61 time for calcul the mask position with numpy : 0.00017690658569335938 nb_pixel_total : 6624 time to create 1 rle with old method : 0.007464170455932617 length of segment : 121 time for calcul the mask position with numpy : 0.000335693359375 nb_pixel_total : 23946 time to create 1 rle with old method : 0.02642369270324707 length of segment : 209 time for calcul the mask position with numpy : 0.0011601448059082031 nb_pixel_total : 75762 time to create 1 rle with old method : 0.07890605926513672 length of segment : 479 time for calcul the mask position with numpy : 0.00011181831359863281 nb_pixel_total : 5851 time to create 1 rle with old method : 0.006749391555786133 length of segment : 86 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 2974 time to create 1 rle with old method : 0.0032999515533447266 length of segment : 69 time for calcul the mask position with numpy : 0.00010824203491210938 nb_pixel_total : 7056 time to create 1 rle with old method : 0.007875442504882812 length of segment : 78 time for calcul the mask position with numpy : 0.0012748241424560547 nb_pixel_total : 89514 time to create 1 rle with old method : 0.09361433982849121 length of segment : 483 time for calcul the mask position with numpy : 0.0001232624053955078 nb_pixel_total : 4949 time to create 1 rle with old method : 0.005318164825439453 length of segment : 107 time for calcul the mask position with numpy : 0.00011849403381347656 nb_pixel_total : 7728 time to create 1 rle with old method : 0.008402585983276367 length of segment : 95 time for calcul the mask position with numpy : 0.0002231597900390625 nb_pixel_total : 13850 time to create 1 rle with old method : 0.015619039535522461 length of segment : 153 time for calcul the mask position with numpy : 0.0001354217529296875 nb_pixel_total : 6427 time to create 1 rle with old method : 0.007644176483154297 length of segment : 97 time for calcul the mask position with numpy : 0.00013399124145507812 nb_pixel_total : 7282 time to create 1 rle with old method : 0.008211135864257812 length of segment : 99 time for calcul the mask position with numpy : 0.0003256797790527344 nb_pixel_total : 14403 time to create 1 rle with old method : 0.019651174545288086 length of segment : 228 time for calcul the mask position with numpy : 0.0003139972686767578 nb_pixel_total : 7387 time to create 1 rle with old method : 0.008104085922241211 length of segment : 95 time for calcul the mask position with numpy : 0.0001285076141357422 nb_pixel_total : 1959 time to create 1 rle with old method : 0.00220489501953125 length of segment : 47 time for calcul the mask position with numpy : 0.0015027523040771484 nb_pixel_total : 21906 time to create 1 rle with old method : 0.024448156356811523 length of segment : 247 time for calcul the mask position with numpy : 0.0002243518829345703 nb_pixel_total : 5952 time to create 1 rle with old method : 0.006846427917480469 length of segment : 89 time for calcul the mask position with numpy : 0.00018262863159179688 nb_pixel_total : 4968 time to create 1 rle with old method : 0.005934715270996094 length of segment : 78 time for calcul the mask position with numpy : 0.00021219253540039062 nb_pixel_total : 5139 time to create 1 rle with old method : 0.006014823913574219 length of segment : 81 time for calcul the mask position with numpy : 0.0004036426544189453 nb_pixel_total : 14010 time to create 1 rle with old method : 0.015705585479736328 length of segment : 156 time for calcul the mask position with numpy : 0.0005390644073486328 nb_pixel_total : 13277 time to create 1 rle with old method : 0.015264511108398438 length of segment : 154 time for calcul the mask position with numpy : 0.0003592967987060547 nb_pixel_total : 11223 time to create 1 rle with old method : 0.013059139251708984 length of segment : 175 time for calcul the mask position with numpy : 0.0005233287811279297 nb_pixel_total : 19709 time to create 1 rle with old method : 0.022604703903198242 length of segment : 164 time for calcul the mask position with numpy : 0.0001316070556640625 nb_pixel_total : 2499 time to create 1 rle with old method : 0.0030367374420166016 length of segment : 44 time for calcul the mask position with numpy : 0.00014090538024902344 nb_pixel_total : 4199 time to create 1 rle with old method : 0.005026102066040039 length of segment : 71 time for calcul the mask position with numpy : 0.0004494190216064453 nb_pixel_total : 6147 time to create 1 rle with old method : 0.008385419845581055 length of segment : 101 time for calcul the mask position with numpy : 0.00030303001403808594 nb_pixel_total : 6428 time to create 1 rle with old method : 0.0077402591705322266 length of segment : 77 time for calcul the mask position with numpy : 0.00041365623474121094 nb_pixel_total : 10806 time to create 1 rle with old method : 0.01241755485534668 length of segment : 146 time for calcul the mask position with numpy : 0.000362396240234375 nb_pixel_total : 9801 time to create 1 rle with old method : 0.011270284652709961 length of segment : 160 time for calcul the mask position with numpy : 0.0004897117614746094 nb_pixel_total : 16150 time to create 1 rle with old method : 0.018466472625732422 length of segment : 151 time for calcul the mask position with numpy : 0.0005009174346923828 nb_pixel_total : 8853 time to create 1 rle with old method : 0.010335206985473633 length of segment : 156 time for calcul the mask position with numpy : 0.0002925395965576172 nb_pixel_total : 5978 time to create 1 rle with old method : 0.0069122314453125 length of segment : 135 time for calcul the mask position with numpy : 0.0006825923919677734 nb_pixel_total : 30360 time to create 1 rle with old method : 0.03439021110534668 length of segment : 339 time for calcul the mask position with numpy : 0.00016307830810546875 nb_pixel_total : 4034 time to create 1 rle with old method : 0.005182981491088867 length of segment : 95 time for calcul the mask position with numpy : 0.000141143798828125 nb_pixel_total : 7882 time to create 1 rle with old method : 0.010757684707641602 length of segment : 86 time for calcul the mask position with numpy : 0.002245187759399414 nb_pixel_total : 116535 time to create 1 rle with old method : 0.12909317016601562 length of segment : 521 time spent for convertir_results : 3.014509677886963 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 61 chid ids of type : 3594 Number RLEs to save : 10779 save missing photos in datou_result : time spend for datou_step_exec : 30.159152507781982 time spend to save output : 0.6781704425811768 total time spend for step 1 : 30.83732295036316 step2:crop_condition Mon Aug 4 13:11:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 19 ! batch 1 Loaded 61 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 26 About to insert : list_path_to_insert length 26 new photo from crops ! About to upload 26 photos upload in portfolio : 3736932 init cache_photo without model_param we have 26 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754305863_3825190 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 26 photos in the portfolio 3736932 time of upload the photos Elapsed time : 7.6893017292022705 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 3736932 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754305872_3825190 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 10 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.2689709663391113 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754305876_3825190 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.7927992343902588 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! About to upload 18 photos upload in portfolio : 3736932 init cache_photo without model_param we have 18 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754305880_3825190 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 18 photos in the portfolio 3736932 time of upload the photos Elapsed time : 4.626229763031006 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3736932 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1754305885_3825190 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 6 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.9492905139923096 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1375112974, 1375112973, 1375112970, 1375112968, 1375112965, 1375112963, 1375112959, 1375112957, 1375112955, 1375112952, 1375112949, 1375112946, 1375112904, 1375112902, 1375112900, 1375112897, 1375112895, 1375112892, 1375112869] Looping around the photos to save general results len do output : 61 /1375116025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1375116127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112974', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112973', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112970', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112968', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112965', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112963', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112959', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112957', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112955', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112952', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112949', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112946', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112904', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112902', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112900', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112897', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112895', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112892', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112869', None, None, None, None, None, '3433890') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 202 time used for this insertion : 0.026709794998168945 save_final save missing photos in datou_result : time spend for datou_step_exec : 26.1416437625885 time spend to save output : 0.02920389175415039 total time spend for step 2 : 26.17084765434265 step3:rle_unique_nms_with_priority Mon Aug 4 13:11:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 61 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 4 nb_hashtags : 4 time to prepare the origin masks : 0.8458492755889893 time for calcul the mask position with numpy : 0.18882989883422852 nb_pixel_total : 1927297 time to create 1 rle with new method : 0.13017606735229492 time for calcul the mask position with numpy : 0.006927013397216797 nb_pixel_total : 5164 time to create 1 rle with old method : 0.005828857421875 time for calcul the mask position with numpy : 0.007051706314086914 nb_pixel_total : 111108 time to create 1 rle with old method : 0.12367534637451172 time for calcul the mask position with numpy : 0.006494045257568359 nb_pixel_total : 7564 time to create 1 rle with old method : 0.00839853286743164 time for calcul the mask position with numpy : 0.00657963752746582 nb_pixel_total : 22467 time to create 1 rle with old method : 0.025596141815185547 create new chi : 0.5239923000335693 time to delete rle : 0.02215576171875 batch 1 Loaded 9 chid ids of type : 3594 +++++++Number RLEs to save : 3098 TO DO : save crop sub photo not yet done ! save time : 0.22603368759155273 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.09922552108764648 time for calcul the mask position with numpy : 0.21027302742004395 nb_pixel_total : 1973793 time to create 1 rle with new method : 0.2435140609741211 time for calcul the mask position with numpy : 0.00600886344909668 nb_pixel_total : 23598 time to create 1 rle with old method : 0.02564859390258789 time for calcul the mask position with numpy : 0.006014347076416016 nb_pixel_total : 6459 time to create 1 rle with old method : 0.007128715515136719 time for calcul the mask position with numpy : 0.006035566329956055 nb_pixel_total : 44652 time to create 1 rle with old method : 0.04664254188537598 time for calcul the mask position with numpy : 0.005831718444824219 nb_pixel_total : 18351 time to create 1 rle with old method : 0.01967763900756836 time for calcul the mask position with numpy : 0.005878448486328125 nb_pixel_total : 6747 time to create 1 rle with old method : 0.0071752071380615234 create new chi : 0.59942626953125 time to delete rle : 0.0004413127899169922 batch 1 Loaded 11 chid ids of type : 3594 +++++++Number RLEs to save : 2894 TO DO : save crop sub photo not yet done ! save time : 0.21248769760131836 No data in photo_id : 1375112970 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.14936518669128418 time for calcul the mask position with numpy : 0.20569896697998047 nb_pixel_total : 1920304 time to create 1 rle with new method : 0.09206628799438477 time for calcul the mask position with numpy : 0.006353855133056641 nb_pixel_total : 111262 time to create 1 rle with old method : 0.11720132827758789 time for calcul the mask position with numpy : 0.005777835845947266 nb_pixel_total : 5565 time to create 1 rle with old method : 0.005884647369384766 time for calcul the mask position with numpy : 0.0062863826751708984 nb_pixel_total : 11252 time to create 1 rle with old method : 0.011801719665527344 time for calcul the mask position with numpy : 0.005682229995727539 nb_pixel_total : 11741 time to create 1 rle with old method : 0.012401342391967773 time for calcul the mask position with numpy : 0.006034135818481445 nb_pixel_total : 6252 time to create 1 rle with old method : 0.0068416595458984375 time for calcul the mask position with numpy : 0.006368160247802734 nb_pixel_total : 7224 time to create 1 rle with old method : 0.00782465934753418 create new chi : 0.5062448978424072 time to delete rle : 0.0005409717559814453 batch 1 Loaded 13 chid ids of type : 3594 +++++++Number RLEs to save : 3272 TO DO : save crop sub photo not yet done ! save time : 0.22617459297180176 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.05042862892150879 time for calcul the mask position with numpy : 0.018186092376708984 nb_pixel_total : 1930454 time to create 1 rle with new method : 0.2055976390838623 time for calcul the mask position with numpy : 0.007581949234008789 nb_pixel_total : 101967 time to create 1 rle with old method : 0.10922598838806152 time for calcul the mask position with numpy : 0.0059642791748046875 nb_pixel_total : 17306 time to create 1 rle with old method : 0.01871967315673828 time for calcul the mask position with numpy : 0.005831003189086914 nb_pixel_total : 23873 time to create 1 rle with old method : 0.02631068229675293 create new chi : 0.40940237045288086 time to delete rle : 0.0004820823669433594 batch 1 Loaded 7 chid ids of type : 3594 +++++++++Number RLEs to save : 3210 TO DO : save crop sub photo not yet done ! save time : 0.23171114921569824 nb_obj : 7 nb_hashtags : 2 time to prepare the origin masks : 0.2835531234741211 time for calcul the mask position with numpy : 0.1970689296722412 nb_pixel_total : 1925499 time to create 1 rle with new method : 0.09489202499389648 time for calcul the mask position with numpy : 0.005933284759521484 nb_pixel_total : 3197 time to create 1 rle with old method : 0.003498077392578125 time for calcul the mask position with numpy : 0.006331920623779297 nb_pixel_total : 5677 time to create 1 rle with old method : 0.00625300407409668 time for calcul the mask position with numpy : 0.006810188293457031 nb_pixel_total : 4708 time to create 1 rle with old method : 0.005093812942504883 time for calcul the mask position with numpy : 0.006935834884643555 nb_pixel_total : 117125 time to create 1 rle with old method : 0.12284970283508301 time for calcul the mask position with numpy : 0.006010293960571289 nb_pixel_total : 6047 time to create 1 rle with old method : 0.006443500518798828 time for calcul the mask position with numpy : 0.0056972503662109375 nb_pixel_total : 5028 time to create 1 rle with old method : 0.005502939224243164 time for calcul the mask position with numpy : 0.005807399749755859 nb_pixel_total : 6319 time to create 1 rle with old method : 0.00676274299621582 create new chi : 0.502521276473999 time to delete rle : 0.0006132125854492188 batch 1 Loaded 15 chid ids of type : 3594 +++++++Number RLEs to save : 3140 TO DO : save crop sub photo not yet done ! save time : 0.2259516716003418 No data in photo_id : 1375112959 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.0647730827331543 time for calcul the mask position with numpy : 0.018505334854125977 nb_pixel_total : 1961442 time to create 1 rle with new method : 0.2094721794128418 time for calcul the mask position with numpy : 0.0060422420501708984 nb_pixel_total : 5851 time to create 1 rle with old method : 0.006388187408447266 time for calcul the mask position with numpy : 0.006464242935180664 nb_pixel_total : 75737 time to create 1 rle with old method : 0.08065247535705566 time for calcul the mask position with numpy : 0.006417751312255859 nb_pixel_total : 23946 time to create 1 rle with old method : 0.026890039443969727 time for calcul the mask position with numpy : 0.00595545768737793 nb_pixel_total : 6624 time to create 1 rle with old method : 0.007341623306274414 create new chi : 0.38152408599853516 time to delete rle : 0.00045228004455566406 batch 1 Loaded 9 chid ids of type : 3594 +++++Number RLEs to save : 2842 TO DO : save crop sub photo not yet done ! save time : 0.20075058937072754 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.2653536796569824 time for calcul the mask position with numpy : 0.18971610069274902 nb_pixel_total : 2063570 time to create 1 rle with new method : 0.10831952095031738 time for calcul the mask position with numpy : 0.00590205192565918 nb_pixel_total : 7056 time to create 1 rle with old method : 0.007721424102783203 time for calcul the mask position with numpy : 0.005995512008666992 nb_pixel_total : 2974 time to create 1 rle with old method : 0.0031747817993164062 create new chi : 0.3314666748046875 time to delete rle : 0.00022983551025390625 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1374 TO DO : save crop sub photo not yet done ! save time : 0.11160540580749512 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.032532453536987305 time for calcul the mask position with numpy : 0.12576866149902344 nb_pixel_total : 1984086 time to create 1 rle with new method : 0.09873414039611816 time for calcul the mask position with numpy : 0.006388664245605469 nb_pixel_total : 89514 time to create 1 rle with old method : 0.09723091125488281 create new chi : 0.336733341217041 time to delete rle : 0.00031495094299316406 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 2046 TO DO : save crop sub photo not yet done ! save time : 0.1619431972503662 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.5218822956085205 time for calcul the mask position with numpy : 0.1815340518951416 nb_pixel_total : 2018961 time to create 1 rle with new method : 0.09859275817871094 time for calcul the mask position with numpy : 0.009404420852661133 nb_pixel_total : 14403 time to create 1 rle with old method : 0.015313148498535156 time for calcul the mask position with numpy : 0.00627446174621582 nb_pixel_total : 7282 time to create 1 rle with old method : 0.00803375244140625 time for calcul the mask position with numpy : 0.005785226821899414 nb_pixel_total : 6427 time to create 1 rle with old method : 0.0069959163665771484 time for calcul the mask position with numpy : 0.0058743953704833984 nb_pixel_total : 13850 time to create 1 rle with old method : 0.014961719512939453 time for calcul the mask position with numpy : 0.006278276443481445 nb_pixel_total : 7728 time to create 1 rle with old method : 0.008700847625732422 time for calcul the mask position with numpy : 0.0062408447265625 nb_pixel_total : 4949 time to create 1 rle with old method : 0.005640745162963867 create new chi : 0.39005208015441895 time to delete rle : 0.0005228519439697266 batch 1 Loaded 13 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 2638 TO DO : save crop sub photo not yet done ! save time : 0.2017810344696045 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.08887600898742676 time for calcul the mask position with numpy : 0.02780747413635254 nb_pixel_total : 2042348 time to create 1 rle with new method : 0.18968772888183594 time for calcul the mask position with numpy : 0.006102085113525391 nb_pixel_total : 21906 time to create 1 rle with old method : 0.023555278778076172 time for calcul the mask position with numpy : 0.0059130191802978516 nb_pixel_total : 1959 time to create 1 rle with old method : 0.002083301544189453 time for calcul the mask position with numpy : 0.0060045719146728516 nb_pixel_total : 7387 time to create 1 rle with old method : 0.008090972900390625 create new chi : 0.2760026454925537 time to delete rle : 0.00029730796813964844 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1858 TO DO : save crop sub photo not yet done ! save time : 0.15099883079528809 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.06501650810241699 time for calcul the mask position with numpy : 0.058054208755493164 nb_pixel_total : 2043531 time to create 1 rle with new method : 0.25861549377441406 time for calcul the mask position with numpy : 0.006055116653442383 nb_pixel_total : 14010 time to create 1 rle with old method : 0.015173912048339844 time for calcul the mask position with numpy : 0.006143093109130859 nb_pixel_total : 5139 time to create 1 rle with old method : 0.00562596321105957 time for calcul the mask position with numpy : 0.006070375442504883 nb_pixel_total : 4968 time to create 1 rle with old method : 0.005612373352050781 time for calcul the mask position with numpy : 0.005968332290649414 nb_pixel_total : 5952 time to create 1 rle with old method : 0.006574153900146484 create new chi : 0.384185791015625 time to delete rle : 0.0003428459167480469 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 1888 TO DO : save crop sub photo not yet done ! save time : 0.14982223510742188 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 0.2729928493499756 time for calcul the mask position with numpy : 0.07351398468017578 nb_pixel_total : 2016546 time to create 1 rle with new method : 0.2314293384552002 time for calcul the mask position with numpy : 0.0060842037200927734 nb_pixel_total : 6147 time to create 1 rle with old method : 0.00681304931640625 time for calcul the mask position with numpy : 0.006131649017333984 nb_pixel_total : 4199 time to create 1 rle with old method : 0.004778146743774414 time for calcul the mask position with numpy : 0.0062177181243896484 nb_pixel_total : 2499 time to create 1 rle with old method : 0.002868175506591797 time for calcul the mask position with numpy : 0.0062296390533447266 nb_pixel_total : 19709 time to create 1 rle with old method : 0.021860837936401367 time for calcul the mask position with numpy : 0.006551027297973633 nb_pixel_total : 11223 time to create 1 rle with old method : 0.012448310852050781 time for calcul the mask position with numpy : 0.0062694549560546875 nb_pixel_total : 13277 time to create 1 rle with old method : 0.014246702194213867 create new chi : 0.4155900478363037 time to delete rle : 0.00042057037353515625 batch 1 Loaded 13 chid ids of type : 3594 ++++++Number RLEs to save : 2498 TO DO : save crop sub photo not yet done ! save time : 0.18320798873901367 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.049182891845703125 time for calcul the mask position with numpy : 0.01901078224182129 nb_pixel_total : 2046565 time to create 1 rle with new method : 0.168442964553833 time for calcul the mask position with numpy : 0.0057468414306640625 nb_pixel_total : 9801 time to create 1 rle with old method : 0.010756731033325195 time for calcul the mask position with numpy : 0.0061986446380615234 nb_pixel_total : 10806 time to create 1 rle with old method : 0.011831045150756836 time for calcul the mask position with numpy : 0.006150245666503906 nb_pixel_total : 6428 time to create 1 rle with old method : 0.006998777389526367 create new chi : 0.23545408248901367 time to delete rle : 0.0003063678741455078 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1846 TO DO : save crop sub photo not yet done ! save time : 0.144730806350708 No data in photo_id : 1375112897 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.03778219223022461 time for calcul the mask position with numpy : 0.01862931251525879 nb_pixel_total : 2057450 time to create 1 rle with new method : 0.057070255279541016 time for calcul the mask position with numpy : 0.0061800479888916016 nb_pixel_total : 16150 time to create 1 rle with old method : 0.017213106155395508 create new chi : 0.09933853149414062 time to delete rle : 0.0002391338348388672 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1382 TO DO : save crop sub photo not yet done ! save time : 0.11274194717407227 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.43575501441955566 time for calcul the mask position with numpy : 0.16344928741455078 nb_pixel_total : 2028409 time to create 1 rle with new method : 0.09808874130249023 time for calcul the mask position with numpy : 0.007170200347900391 nb_pixel_total : 30360 time to create 1 rle with old method : 0.03290987014770508 time for calcul the mask position with numpy : 0.00587153434753418 nb_pixel_total : 5978 time to create 1 rle with old method : 0.006482124328613281 time for calcul the mask position with numpy : 0.005906343460083008 nb_pixel_total : 8853 time to create 1 rle with old method : 0.009582757949829102 create new chi : 0.34147191047668457 time to delete rle : 0.0004665851593017578 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 2340 TO DO : save crop sub photo not yet done ! save time : 0.17131543159484863 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 0.22633004188537598 time for calcul the mask position with numpy : 0.16323614120483398 nb_pixel_total : 1945149 time to create 1 rle with new method : 0.0929727554321289 time for calcul the mask position with numpy : 0.0068552494049072266 nb_pixel_total : 116535 time to create 1 rle with old method : 0.12447381019592285 time for calcul the mask position with numpy : 0.006253480911254883 nb_pixel_total : 7882 time to create 1 rle with old method : 0.008280754089355469 time for calcul the mask position with numpy : 0.006012916564941406 nb_pixel_total : 4034 time to create 1 rle with old method : 0.0044994354248046875 create new chi : 0.42315673828125 time to delete rle : 0.0004436969757080078 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 2484 TO DO : save crop sub photo not yet done ! save time : 0.18590378761291504 map_output_result : {1375112974: (0.0, 'Should be the crop_list due to order', 0), 1375112973: (0.0, 'Should be the crop_list due to order', 0), 1375112970: (0.0, 'Should be the crop_list due to order', 0.0), 1375112968: (0.0, 'Should be the crop_list due to order', 0), 1375112965: (0.0, 'Should be the crop_list due to order', 0), 1375112963: (0.0, 'Should be the crop_list due to order', 0), 1375112959: (0.0, 'Should be the crop_list due to order', 0.0), 1375112957: (0.0, 'Should be the crop_list due to order', 0), 1375112955: (0.0, 'Should be the crop_list due to order', 0), 1375112952: (0.0, 'Should be the crop_list due to order', 0), 1375112949: (0.0, 'Should be the crop_list due to order', 0), 1375112946: (0.0, 'Should be the crop_list due to order', 0), 1375112904: (0.0, 'Should be the crop_list due to order', 0), 1375112902: (0.0, 'Should be the crop_list due to order', 0), 1375112900: (0.0, 'Should be the crop_list due to order', 0), 1375112897: (0.0, 'Should be the crop_list due to order', 0.0), 1375112895: (0.0, 'Should be the crop_list due to order', 0), 1375112892: (0.0, 'Should be the crop_list due to order', 0), 1375112869: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1375112974, 1375112973, 1375112970, 1375112968, 1375112965, 1375112963, 1375112959, 1375112957, 1375112955, 1375112952, 1375112949, 1375112946, 1375112904, 1375112902, 1375112900, 1375112897, 1375112895, 1375112892, 1375112869] Looping around the photos to save general results len do output : 19 /1375112974.Didn't retrieve data . /1375112973.Didn't retrieve data . /1375112970.Didn't retrieve data . /1375112968.Didn't retrieve data . /1375112965.Didn't retrieve data . /1375112963.Didn't retrieve data . /1375112959.Didn't retrieve data . /1375112957.Didn't retrieve data . /1375112955.Didn't retrieve data . /1375112952.Didn't retrieve data . /1375112949.Didn't retrieve data . /1375112946.Didn't retrieve data . /1375112904.Didn't retrieve data . /1375112902.Didn't retrieve data . /1375112900.Didn't retrieve data . /1375112897.Didn't retrieve data . /1375112895.Didn't retrieve data . /1375112892.Didn't retrieve data . /1375112869.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112974', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112973', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112970', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112968', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112965', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112963', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112959', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112957', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112955', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112952', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112949', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112946', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112904', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112902', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112900', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112897', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112895', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112892', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112869', None, None, None, None, None, '3433890') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 57 time used for this insertion : 0.014253854751586914 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.108603239059448 time spend to save output : 0.015028715133666992 total time spend for step 3 : 13.123631954193115 step4:ventilate_hashtags_in_portfolio Mon Aug 4 13:11:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 25638289 get user id for portfolio 25638289 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25638289 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_clair','environnement','carton','autre','flou','pehd','mal_croppe','background','metal','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25638289 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_clair','environnement','carton','autre','flou','pehd','mal_croppe','background','metal','pet_fonce')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25638289 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','pet_clair','environnement','carton','autre','flou','pehd','mal_croppe','background','metal','pet_fonce')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/25638380,25638381,25638382,25638383,25638384,25638385,25638386,25638387,25638388,25638389,25638390?tags=papier,pet_clair,environnement,carton,autre,flou,pehd,mal_croppe,background,metal,pet_fonce Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1375112974, 1375112973, 1375112970, 1375112968, 1375112965, 1375112963, 1375112959, 1375112957, 1375112955, 1375112952, 1375112949, 1375112946, 1375112904, 1375112902, 1375112900, 1375112897, 1375112895, 1375112892, 1375112869] Looping around the photos to save general results len do output : 1 /25638289. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112974', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112973', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112970', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112968', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112965', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112963', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112959', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112957', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112955', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112952', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112949', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112946', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112904', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112902', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112900', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112897', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112895', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112892', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112869', None, None, None, None, None, '3433890') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.017372846603393555 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.7008178234100342 time spend to save output : 0.017680883407592773 total time spend for step 4 : 0.718498706817627 step5:final Mon Aug 4 13:11:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1375112974: ('0.03279818469785575',), 1375112973: ('0.03279818469785575',), 1375112970: ('0.03279818469785575',), 1375112968: ('0.03279818469785575',), 1375112965: ('0.03279818469785575',), 1375112963: ('0.03279818469785575',), 1375112959: ('0.03279818469785575',), 1375112957: ('0.03279818469785575',), 1375112955: ('0.03279818469785575',), 1375112952: ('0.03279818469785575',), 1375112949: ('0.03279818469785575',), 1375112946: ('0.03279818469785575',), 1375112904: ('0.03279818469785575',), 1375112902: ('0.03279818469785575',), 1375112900: ('0.03279818469785575',), 1375112897: ('0.03279818469785575',), 1375112895: ('0.03279818469785575',), 1375112892: ('0.03279818469785575',), 1375112869: ('0.03279818469785575',)} new output for save of step final : {1375112974: ('0.03279818469785575',), 1375112973: ('0.03279818469785575',), 1375112970: ('0.03279818469785575',), 1375112968: ('0.03279818469785575',), 1375112965: ('0.03279818469785575',), 1375112963: ('0.03279818469785575',), 1375112959: ('0.03279818469785575',), 1375112957: ('0.03279818469785575',), 1375112955: ('0.03279818469785575',), 1375112952: ('0.03279818469785575',), 1375112949: ('0.03279818469785575',), 1375112946: ('0.03279818469785575',), 1375112904: ('0.03279818469785575',), 1375112902: ('0.03279818469785575',), 1375112900: ('0.03279818469785575',), 1375112897: ('0.03279818469785575',), 1375112895: ('0.03279818469785575',), 1375112892: ('0.03279818469785575',), 1375112869: ('0.03279818469785575',)} [1375112974, 1375112973, 1375112970, 1375112968, 1375112965, 1375112963, 1375112959, 1375112957, 1375112955, 1375112952, 1375112949, 1375112946, 1375112904, 1375112902, 1375112900, 1375112897, 1375112895, 1375112892, 1375112869] Looping around the photos to save general results len do output : 19 /1375112974.Didn't retrieve data . /1375112973.Didn't retrieve data . /1375112970.Didn't retrieve data . /1375112968.Didn't retrieve data . /1375112965.Didn't retrieve data . /1375112963.Didn't retrieve data . /1375112959.Didn't retrieve data . /1375112957.Didn't retrieve data . /1375112955.Didn't retrieve data . /1375112952.Didn't retrieve data . /1375112949.Didn't retrieve data . /1375112946.Didn't retrieve data . /1375112904.Didn't retrieve data . /1375112902.Didn't retrieve data . /1375112900.Didn't retrieve data . /1375112897.Didn't retrieve data . /1375112895.Didn't retrieve data . /1375112892.Didn't retrieve data . /1375112869.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112974', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112973', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112970', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112968', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112965', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112963', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112959', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112957', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112955', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112952', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112949', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112946', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112904', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112902', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112900', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112897', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112895', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112892', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112869', None, None, None, None, None, '3433890') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 57 time used for this insertion : 0.013872385025024414 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1292116641998291 time spend to save output : 0.01467442512512207 total time spend for step 5 : 0.14388608932495117 step6:blur_detection Mon Aug 4 13:11:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b.jpg resize: (1080, 1920) 1375112974 -4.58765295247935 treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566.jpg resize: (1080, 1920) 1375112973 0.774386223290712 treat image : temp/1754305828_3825190_1375112970_5241c5c990d9e4454614c1543aa6d354.jpg resize: (1080, 1920) 1375112970 -0.6301844740161293 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a.jpg resize: (1080, 1920) 1375112968 0.40449288928639254 treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646.jpg resize: (1080, 1920) 1375112965 -0.888365310730589 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474.jpg resize: (1080, 1920) 1375112963 -4.716151318674096 treat image : temp/1754305828_3825190_1375112959_03bf42faafa092e10e06033e2f35cf67.jpg resize: (1080, 1920) 1375112959 -1.0263184602418618 treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d.jpg resize: (1080, 1920) 1375112957 -0.7745859875082024 treat image : temp/1754305828_3825190_1375112955_9889c141d15bc898431581289a8bbadf.jpg resize: (1080, 1920) 1375112955 -1.1667025591840177 treat image : temp/1754305828_3825190_1375112952_55a19d2d2b4b376e9b0d7c3c25508dd2.jpg resize: (1080, 1920) 1375112952 -0.7882777953009885 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef.jpg resize: (1080, 1920) 1375112949 -5.153377196450694 treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871.jpg resize: (1080, 1920) 1375112946 -1.6665112092226113 treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613.jpg resize: (1080, 1920) 1375112904 -0.2708085531180801 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a.jpg resize: (1080, 1920) 1375112902 -0.3263533855957249 treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e.jpg resize: (1080, 1920) 1375112900 0.2695841563703895 treat image : temp/1754305828_3825190_1375112897_1acced597f438926af6a9bace6e216fd.jpg resize: (1080, 1920) 1375112897 0.13560870810939113 treat image : temp/1754305828_3825190_1375112895_959214c5af70205ea6adcfb1404b1c35.jpg resize: (1080, 1920) 1375112895 0.736743925200863 treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20.jpg resize: (1080, 1920) 1375112892 0.6127774531929904 treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88.jpg resize: (1080, 1920) 1375112869 2.1033544668640936 treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539927_0.png resize: (210, 205) 1375116025 -4.83812826900108 treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539931_0.png resize: (75, 118) 1375116026 -1.6610240219860222 treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539933_0.png resize: (265, 258) 1375116027 -2.814831362401452 treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539934_0.png resize: (86, 116) 1375116028 -0.9826565961813968 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539936_0.png resize: (62, 160) 1375116029 -2.67628305420106 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539937_0.png resize: (93, 105) 1375116030 0.12034185058199139 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539940_0.png resize: (71, 127) 1375116031 1.7351024925696954 treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646_rle_crop_3902539943_0.png resize: (169, 242) 1375116032 -3.010264938773458 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539945_0.png resize: (67, 111) 1375116033 -3.840433008300232 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539946_0.png resize: (83, 84) 1375116034 -4.6084305573246525 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539947_0.png resize: (134, 77) 1375116035 -4.585761138912465 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539949_0.png resize: (95, 67) 1375116036 -4.5543324671347145 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539950_0.png resize: (50, 141) 1375116037 -4.556486042525506 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539951_0.png resize: (61, 74) 1375116038 -3.6339071996873806 treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539952_0.png resize: (165, 109) 1375116039 -1.076184588157002 treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539955_0.png resize: (82, 95) 1375116040 -3.6830882530384232 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539959_0.png resize: (106, 86) 1375116041 -2.0849750077397315 treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871_rle_crop_3902539965_0.png resize: (95, 124) 1375116042 -1.1941806461857645 treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539968_0.png resize: (89, 105) 1375116043 0.40297890008334014 treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539969_0.png resize: (78, 99) 1375116044 -3.130727900538277 treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539970_0.png resize: (80, 110) 1375116045 -2.652997613213928 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539974_0.png resize: (157, 196) 1375116046 -1.969100877695896 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539975_0.png resize: (41, 73) 1375116047 -1.1937243141570926 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539976_0.png resize: (67, 93) 1375116048 -1.1887822280531894 treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20_rle_crop_3902539982_0.png resize: (148, 111) 1375116049 -1.2723702769629064 treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88_rle_crop_3902539986_0.png resize: (84, 102) 1375116050 20.0 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539938_0.png resize: (96, 213) 1375116069 -2.75943001939133 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539941_0.png resize: (536, 345) 1375116071 -0.05678396089157235 treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539948_0.png resize: (534, 363) 1375116072 0.0366815913913116 treat image : temp/1754305828_3825190_1375112955_9889c141d15bc898431581289a8bbadf_rle_crop_3902539956_0.png resize: (68, 53) 1375116073 3.381476556623061 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539960_0.png resize: (94, 96) 1375116075 -2.7236885686766406 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539963_0.png resize: (99, 129) 1375116076 -4.680254021250225 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539977_0.png resize: (100, 78) 1375116077 1.1228856581138786 treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e_rle_crop_3902539978_0.png resize: (77, 125) 1375116079 0.1420267324484415 treat image : temp/1754305828_3825190_1375112895_959214c5af70205ea6adcfb1404b1c35_rle_crop_3902539981_0.png resize: (148, 136) 1375116080 0.6295019668089866 treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20_rle_crop_3902539983_0.png resize: (135, 66) 1375116081 -1.5243390304568283 treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539928_0.png resize: (70, 137) 1375116090 -2.212592877732668 treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539929_0.png resize: (522, 346) 1375116103 0.3860795633491528 treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539935_0.png resize: (272, 129) 1375116104 -1.520170052988745 treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646_rle_crop_3902539942_0.png resize: (214, 192) 1375116105 -1.9947769104692439 treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646_rle_crop_3902539944_0.png resize: (554, 322) 1375116106 -0.3182133478147876 treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539953_0.png resize: (209, 170) 1375116107 -1.0574315835828143 treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539954_0.png resize: (481, 337) 1375116108 -0.41362065982249907 treat image : temp/1754305828_3825190_1375112952_55a19d2d2b4b376e9b0d7c3c25508dd2_rle_crop_3902539958_0.png resize: (515, 363) 1375116109 -0.36370440912006563 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539961_0.png resize: (141, 123) 1375116110 -2.566978289839337 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539962_0.png resize: (77, 122) 1375116111 -4.986569650961489 treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539964_0.png resize: (151, 196) 1375116112 -4.7320556157653195 treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871_rle_crop_3902539966_0.png resize: (47, 53) 1375116113 -1.819789567178073 treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871_rle_crop_3902539967_0.png resize: (247, 205) 1375116114 -1.587505343166106 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539972_0.png resize: (154, 136) 1375116115 -1.6704994332146603 treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539973_0.png resize: (127, 131) 1375116116 -1.632606452619206 treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e_rle_crop_3902539979_0.png resize: (146, 106) 1375116117 -1.1823553077908968 treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e_rle_crop_3902539980_0.png resize: (159, 88) 1375116118 -0.500888922162661 treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20_rle_crop_3902539984_0.png resize: (331, 116) 1375116119 -0.10756419371578324 treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88_rle_crop_3902539987_0.png resize: (517, 378) 1375116120 0.24834656021042922 treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539930_0.png resize: (128, 56) 1375116121 -2.3688056332003984 treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539932_0.png resize: (153, 167) 1375116122 -1.4699622036265663 treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539939_0.png resize: (120, 134) 1375116123 -1.1403686822726768 treat image : temp/1754305828_3825190_1375112955_9889c141d15bc898431581289a8bbadf_rle_crop_3902539957_0.png resize: (77, 102) 1375116124 1.5988751372398828 treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539971_0.png resize: (150, 136) 1375116125 -2.551158646378036 treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88_rle_crop_3902539985_0.png resize: (112, 53) 1375116127 -1.69922616379399 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 80 time used for this insertion : 0.01523590087890625 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 80 time used for this insertion : 0.014764785766601562 save missing photos in datou_result : time spend for datou_step_exec : 15.22663688659668 time spend to save output : 0.03583979606628418 total time spend for step 6 : 15.262476682662964 step7:brightness Mon Aug 4 13:11:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b.jpg treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566.jpg treat image : temp/1754305828_3825190_1375112970_5241c5c990d9e4454614c1543aa6d354.jpg treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a.jpg treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646.jpg treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474.jpg treat image : temp/1754305828_3825190_1375112959_03bf42faafa092e10e06033e2f35cf67.jpg treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d.jpg treat image : temp/1754305828_3825190_1375112955_9889c141d15bc898431581289a8bbadf.jpg treat image : temp/1754305828_3825190_1375112952_55a19d2d2b4b376e9b0d7c3c25508dd2.jpg treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef.jpg treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871.jpg treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613.jpg treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a.jpg treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e.jpg treat image : temp/1754305828_3825190_1375112897_1acced597f438926af6a9bace6e216fd.jpg treat image : temp/1754305828_3825190_1375112895_959214c5af70205ea6adcfb1404b1c35.jpg treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20.jpg treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88.jpg treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539927_0.png treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539931_0.png treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539933_0.png treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539934_0.png treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539936_0.png treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539937_0.png treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539940_0.png treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646_rle_crop_3902539943_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539945_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539946_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539947_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539949_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539950_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539951_0.png treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539952_0.png treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539955_0.png treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539959_0.png treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871_rle_crop_3902539965_0.png treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539968_0.png treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539969_0.png treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539970_0.png treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539974_0.png treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539975_0.png treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539976_0.png treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20_rle_crop_3902539982_0.png treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88_rle_crop_3902539986_0.png treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539938_0.png treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539941_0.png treat image : temp/1754305828_3825190_1375112963_06532293cac7d0c155ba9a750d6d7474_rle_crop_3902539948_0.png treat image : temp/1754305828_3825190_1375112955_9889c141d15bc898431581289a8bbadf_rle_crop_3902539956_0.png treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539960_0.png treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539963_0.png treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539977_0.png treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e_rle_crop_3902539978_0.png treat image : temp/1754305828_3825190_1375112895_959214c5af70205ea6adcfb1404b1c35_rle_crop_3902539981_0.png treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20_rle_crop_3902539983_0.png treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539928_0.png treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539929_0.png treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539935_0.png treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646_rle_crop_3902539942_0.png treat image : temp/1754305828_3825190_1375112965_b7d8cedf966d207ee8611a64dcc52646_rle_crop_3902539944_0.png treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539953_0.png treat image : temp/1754305828_3825190_1375112957_24feefad7999d79a8df42eff00ce9b1d_rle_crop_3902539954_0.png treat image : temp/1754305828_3825190_1375112952_55a19d2d2b4b376e9b0d7c3c25508dd2_rle_crop_3902539958_0.png treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539961_0.png treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539962_0.png treat image : temp/1754305828_3825190_1375112949_b75742d7aa6887d8c3b29877e47c61ef_rle_crop_3902539964_0.png treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871_rle_crop_3902539966_0.png treat image : temp/1754305828_3825190_1375112946_86a72836ed321aa06f20833f8b8a6871_rle_crop_3902539967_0.png treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539972_0.png treat image : temp/1754305828_3825190_1375112902_eaa3ccc8b2be614ef9913c12c2cfb49a_rle_crop_3902539973_0.png treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e_rle_crop_3902539979_0.png treat image : temp/1754305828_3825190_1375112900_e850099fa0557408fbda05e742abcf8e_rle_crop_3902539980_0.png treat image : temp/1754305828_3825190_1375112892_e40a050a5dea6fa2e80f41fd53cc1a20_rle_crop_3902539984_0.png treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88_rle_crop_3902539987_0.png treat image : temp/1754305828_3825190_1375112974_6697648a1b4ef8e1859831d3fe9c473b_rle_crop_3902539930_0.png treat image : temp/1754305828_3825190_1375112973_5ee29cb29957002671126dd6a6cb6566_rle_crop_3902539932_0.png treat image : temp/1754305828_3825190_1375112968_995bc9fe84092cc20173159a102b6b1a_rle_crop_3902539939_0.png treat image : temp/1754305828_3825190_1375112955_9889c141d15bc898431581289a8bbadf_rle_crop_3902539957_0.png treat image : temp/1754305828_3825190_1375112904_66502a7302f70c2cbd4d9626f8047613_rle_crop_3902539971_0.png treat image : temp/1754305828_3825190_1375112869_d572d855fcc44a1f2ac098f26e90ce88_rle_crop_3902539985_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 80 time used for this insertion : 0.016863107681274414 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 80 time used for this insertion : 0.017455339431762695 save missing photos in datou_result : time spend for datou_step_exec : 4.282339334487915 time spend to save output : 0.038843631744384766 total time spend for step 7 : 4.3211829662323 step8:velours_tree Mon Aug 4 13:12:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.12360501289367676 time spend to save output : 3.147125244140625e-05 total time spend for step 8 : 0.12363648414611816 step9:send_mail_cod Mon Aug 4 13:12:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P25638289_04-08-2025_13_12_01.pdf 25638380 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette256383801754305921 25638381 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette256383811754305922 25638383 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette256383831754305923 25638384 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette256383841754305924 25638385 imagette256383851754305924 25638386 imagette256383861754305924 25638387 imagette256383871754305924 25638388 imagette256383881754305924 25638389 change filename to text .imagette256383891754305924 25638390 imagette256383901754305924 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=25638289 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/25638380,25638381,25638382,25638383,25638384,25638385,25638386,25638387,25638388,25638389,25638390?tags=papier,pet_clair,environnement,carton,autre,flou,pehd,mal_croppe,background,metal,pet_fonce args[1375112974] : ((1375112974, -4.58765295247935, 492609224), (1375112974, 0.35815096977499167, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112973] : ((1375112973, 0.774386223290712, 492688767), (1375112973, 0.48510940586707807, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112970] : ((1375112970, -0.6301844740161293, 492688767), (1375112970, 0.8024089149631963, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112968] : ((1375112968, 0.40449288928639254, 492688767), (1375112968, 0.2107919997029653, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112965] : ((1375112965, -0.888365310730589, 492688767), (1375112965, 0.45386601041348923, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112963] : ((1375112963, -4.716151318674096, 492609224), (1375112963, 0.23860303736520522, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112959] : ((1375112959, -1.0263184602418618, 492688767), (1375112959, 0.7800856791582907, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112957] : ((1375112957, -0.7745859875082024, 492688767), (1375112957, 0.7669619440964051, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112955] : ((1375112955, -1.1667025591840177, 492688767), (1375112955, 0.8749958037750055, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112952] : ((1375112952, -0.7882777953009885, 492688767), (1375112952, 0.7009944195778409, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112949] : ((1375112949, -5.153377196450694, 492609224), (1375112949, 0.29600917803804705, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112946] : ((1375112946, -1.6665112092226113, 492688767), (1375112946, 0.61736308279658, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112904] : ((1375112904, -0.2708085531180801, 492688767), (1375112904, 0.7131277496827816, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112902] : ((1375112902, -0.3263533855957249, 492688767), (1375112902, 0.7030895836331316, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112900] : ((1375112900, 0.2695841563703895, 492688767), (1375112900, 0.578056011527643, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112897] : ((1375112897, 0.13560870810939113, 492688767), (1375112897, 0.7448470671590673, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112895] : ((1375112895, 0.736743925200863, 492688767), (1375112895, 0.6602420566813912, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112892] : ((1375112892, 0.6127774531929904, 492688767), (1375112892, 0.6997849024554291, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com args[1375112869] : ((1375112869, 2.1033544668640936, 492688767), (1375112869, 0.304439190795896, 2107752395), '0.03279818469785575') We are sending mail with results at report@fotonower.com refus_total : 0.03279818469785575 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25638289 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25638289_04-08-2025_13_12_01.pdf results_Auto_P25638289_04-08-2025_13_12_01.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25638289_04-08-2025_13_12_01.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','25638289','results_Auto_P25638289_04-08-2025_13_12_01.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25638289_04-08-2025_13_12_01.pdf','pdf','','0.32','0.03279818469785575') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/25638289

https://www.fotonower.com/image?json=false&list_photos_id=1375112974
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112973
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112970
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112968
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112965
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112963
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112959
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112957
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112955
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112952
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112949
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112946
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112904
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112902
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112900
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112897
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112895
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112892
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1375112869
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.1033544668640936)

Dans ces conditions,le taux de refus est: 3.28%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/25638380?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/25638381?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/25638383?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/25638384?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/25638389?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25638289_04-08-2025_13_12_01.pdf.

Lien vers velours :https://www.fotonower.com/velours/25638380,25638381,25638382,25638383,25638384,25638385,25638386,25638387,25638388,25638389,25638390?tags=papier,pet_clair,environnement,carton,autre,flou,pehd,mal_croppe,background,metal,pet_fonce.


L'équipe Fotonower 202 b'' Server: nginx Date: Mon, 04 Aug 2025 11:12:06 GMT Content-Length: 0 Connection: close X-Message-Id: i00871vbR9iT1KkmniQlvg Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1375112974, 1375112973, 1375112970, 1375112968, 1375112965, 1375112963, 1375112959, 1375112957, 1375112955, 1375112952, 1375112949, 1375112946, 1375112904, 1375112902, 1375112900, 1375112897, 1375112895, 1375112892, 1375112869] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112974', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112973', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112970', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112968', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112965', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112963', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112959', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112957', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112955', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112952', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112949', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112946', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112904', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112902', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112900', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112897', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112895', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112892', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112869', None, None, None, None, None, '3433890') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.017428159713745117 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.5039074420928955 time spend to save output : 0.017661333084106445 total time spend for step 9 : 5.521568775177002 step10:split_time_score Mon Aug 4 13:12:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('12', 19),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 04082025 25638289 Nombre de photos uploadées : 19 / 23040 (0%) 04082025 25638289 Nombre de photos taguées (types de déchets): 0 / 19 (0%) 04082025 25638289 Nombre de photos taguées (volume) : 0 / 19 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 5.7220458984375e-06 ??????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0008893013000488281 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.21884512901306152 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.020633053626543208 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25630909_04-08-2025_08_41_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25630909 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25630909 AND mptpi.`type`=3594 To do Qualite : 0.08988558545524694 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25632840_04-08-2025_10_01_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25632840 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25632840 AND mptpi.`type`=3594 To do Qualite : 0.26266080729166663 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25636092_04-08-2025_12_12_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25636092 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25636092 AND mptpi.`type`=3594 To do Qualite : 0.26008359053497937 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25636094_04-08-2025_11_52_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25636094 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25636094 AND mptpi.`type`=3594 To do Qualite : 0.031987895447530876 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25636131_04-08-2025_11_41_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25636131 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25636131 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25638279 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25638287 order by id desc limit 1 Qualite : 0.03279818469785575 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25638289_04-08-2025_13_12_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25638289 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25638289 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'04082025': {'nb_upload': 19, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1375112974, 1375112973, 1375112970, 1375112968, 1375112965, 1375112963, 1375112959, 1375112957, 1375112955, 1375112952, 1375112949, 1375112946, 1375112904, 1375112902, 1375112900, 1375112897, 1375112895, 1375112892, 1375112869] Looping around the photos to save general results len do output : 1 /25638289Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112974', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112973', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112970', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112968', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112965', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112963', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112959', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112957', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112955', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112952', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112949', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112946', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112904', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112902', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112900', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112897', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112895', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112892', None, None, None, None, None, '3433890') ('3318', None, None, None, None, None, None, None, '3433890') ('3318', '25638289', '1375112869', None, None, None, None, None, '3433890') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.015584945678710938 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.6182408332824707 time spend to save output : 0.015849828720092773 total time spend for step 10 : 2.6340906620025635 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 19 set_done_treatment 51.97user 26.77system 1:44.74elapsed 75%CPU (0avgtext+0avgdata 3009580maxresident)k 504576inputs+23088outputs (61major+2093322minor)pagefaults 0swaps