python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 451795 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['3795462'] with mtr_portfolio_ids : ['27358921'] and first list_photo_ids : [] new path : /proc/451795/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 22 ; length of list_pids : 22 ; length of list_args : 22 time to download the photos : 3.381162166595459 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Tue Sep 30 18:00:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10518 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-30 18:00:35.346234: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-30 18:00:35.372607: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-30 18:00:35.375028: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f88e0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-30 18:00:35.375106: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-30 18:00:35.379868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-30 18:00:35.507978: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x39ca85f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-30 18:00:35.508056: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-30 18:00:35.509505: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-30 18:00:35.509926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-30 18:00:35.512810: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-30 18:00:35.515355: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-30 18:00:35.515942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-30 18:00:35.519210: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-30 18:00:35.520811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-30 18:00:35.527667: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-30 18:00:35.529399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-30 18:00:35.529501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-30 18:00:35.530394: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-30 18:00:35.530421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-30 18:00:35.530434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-30 18:00:35.532223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8413 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-30 18:00:35.976471: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-30 18:00:35.976573: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-30 18:00:35.976601: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-30 18:00:35.976627: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-30 18:00:35.976651: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-30 18:00:35.976675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-30 18:00:35.976699: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-30 18:00:35.976723: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-30 18:00:35.978131: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-30 18:00:35.979753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-30 18:00:35.979827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-30 18:00:35.979860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-30 18:00:35.979888: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-30 18:00:35.979920: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-30 18:00:35.979952: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-30 18:00:35.979982: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-30 18:00:35.980023: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-30 18:00:35.981275: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-30 18:00:35.981313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-30 18:00:35.981323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-30 18:00:35.981331: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-30 18:00:35.982548: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8413 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-30 18:00:44.463469: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-30 18:00:44.803995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-30 18:00:48.040062: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:48.041129: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:48.041812: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:48.934316: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:48.934756: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-09-30 18:00:49.006947: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.007169: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.008558: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.008580: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.015031: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.015054: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.015768: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.015783: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.087747: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.087801: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.088529: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.088546: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.089323: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.089339: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.090102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.090118: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.098189: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.098215: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 417.62MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.098982: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.098998: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 417.62MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-09-30 18:00:49.125672: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.126682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.127659: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.128659: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.132929: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.133704: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.134478: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.135259: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.137934: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.137955: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-09-30 18:00:49.146571: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.147351: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.156506: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.157287: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.158027: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.158796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.159568: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-09-30 18:00:49.160352: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 22 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 24.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 28.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 27.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 15.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 22.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 31.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 21.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 18.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 20.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (1080, 1920, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 1920.00000 nb d'objets trouves : 6 Detection mask done ! Trying to reset tf kernel 452715 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1119 tf kernel not reseted sub process len(results) : 22 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 22 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2495 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0008490085601806641 nb_pixel_total : 6144 time to create 1 rle with old method : 0.010541439056396484 length of segment : 88 time for calcul the mask position with numpy : 0.0037157535552978516 nb_pixel_total : 112565 time to create 1 rle with old method : 0.16263413429260254 length of segment : 578 time for calcul the mask position with numpy : 0.013317108154296875 nb_pixel_total : 716687 time to create 1 rle with new method : 0.05359220504760742 length of segment : 955 time for calcul the mask position with numpy : 0.0006067752838134766 nb_pixel_total : 33388 time to create 1 rle with old method : 0.0433499813079834 length of segment : 194 time for calcul the mask position with numpy : 0.0002391338348388672 nb_pixel_total : 5774 time to create 1 rle with old method : 0.00822591781616211 length of segment : 79 time for calcul the mask position with numpy : 0.00021958351135253906 nb_pixel_total : 5821 time to create 1 rle with old method : 0.007898330688476562 length of segment : 87 time for calcul the mask position with numpy : 0.002789020538330078 nb_pixel_total : 117569 time to create 1 rle with old method : 0.14477014541625977 length of segment : 505 time for calcul the mask position with numpy : 0.0026519298553466797 nb_pixel_total : 119582 time to create 1 rle with old method : 0.13940739631652832 length of segment : 523 time for calcul the mask position with numpy : 0.0028018951416015625 nb_pixel_total : 121274 time to create 1 rle with old method : 0.16031932830810547 length of segment : 575 time for calcul the mask position with numpy : 0.002440929412841797 nb_pixel_total : 106099 time to create 1 rle with old method : 0.12720799446105957 length of segment : 662 time for calcul the mask position with numpy : 0.0003268718719482422 nb_pixel_total : 6189 time to create 1 rle with old method : 0.007400989532470703 length of segment : 118 time for calcul the mask position with numpy : 0.00040459632873535156 nb_pixel_total : 11116 time to create 1 rle with old method : 0.012992143630981445 length of segment : 187 time for calcul the mask position with numpy : 0.00012969970703125 nb_pixel_total : 2363 time to create 1 rle with old method : 0.0030121803283691406 length of segment : 49 time for calcul the mask position with numpy : 0.0001919269561767578 nb_pixel_total : 4643 time to create 1 rle with old method : 0.005579471588134766 length of segment : 77 time for calcul the mask position with numpy : 0.0022606849670410156 nb_pixel_total : 111082 time to create 1 rle with old method : 0.12550020217895508 length of segment : 535 time for calcul the mask position with numpy : 0.22215580940246582 nb_pixel_total : 767990 time to create 1 rle with new method : 0.06106305122375488 length of segment : 987 time for calcul the mask position with numpy : 0.000331878662109375 nb_pixel_total : 10485 time to create 1 rle with old method : 0.013173103332519531 length of segment : 158 time for calcul the mask position with numpy : 0.000850677490234375 nb_pixel_total : 19779 time to create 1 rle with old method : 0.023875951766967773 length of segment : 190 time for calcul the mask position with numpy : 0.0002930164337158203 nb_pixel_total : 6056 time to create 1 rle with old method : 0.007230043411254883 length of segment : 89 time for calcul the mask position with numpy : 0.0006530284881591797 nb_pixel_total : 20457 time to create 1 rle with old method : 0.026297807693481445 length of segment : 140 time for calcul the mask position with numpy : 0.3597075939178467 nb_pixel_total : 1524388 time to create 1 rle with new method : 1.2507476806640625 length of segment : 1094 time for calcul the mask position with numpy : 0.0003173351287841797 nb_pixel_total : 14643 time to create 1 rle with old method : 0.01684284210205078 length of segment : 181 time for calcul the mask position with numpy : 7.987022399902344e-05 nb_pixel_total : 2332 time to create 1 rle with old method : 0.002928495407104492 length of segment : 54 time for calcul the mask position with numpy : 0.0002410411834716797 nb_pixel_total : 5324 time to create 1 rle with old method : 0.0063555240631103516 length of segment : 92 time for calcul the mask position with numpy : 0.0006444454193115234 nb_pixel_total : 26670 time to create 1 rle with old method : 0.030936241149902344 length of segment : 178 time for calcul the mask position with numpy : 0.00025725364685058594 nb_pixel_total : 4479 time to create 1 rle with old method : 0.005377054214477539 length of segment : 98 time for calcul the mask position with numpy : 0.00036072731018066406 nb_pixel_total : 10262 time to create 1 rle with old method : 0.012289285659790039 length of segment : 108 time for calcul the mask position with numpy : 0.00023245811462402344 nb_pixel_total : 4860 time to create 1 rle with old method : 0.0058748722076416016 length of segment : 100 time for calcul the mask position with numpy : 0.0021126270294189453 nb_pixel_total : 101841 time to create 1 rle with old method : 0.11471796035766602 length of segment : 519 time for calcul the mask position with numpy : 0.00017452239990234375 nb_pixel_total : 5971 time to create 1 rle with old method : 0.007175922393798828 length of segment : 83 time for calcul the mask position with numpy : 0.00025010108947753906 nb_pixel_total : 5051 time to create 1 rle with old method : 0.006133317947387695 length of segment : 102 time for calcul the mask position with numpy : 0.00020313262939453125 nb_pixel_total : 6132 time to create 1 rle with old method : 0.0075380802154541016 length of segment : 87 time for calcul the mask position with numpy : 0.002402782440185547 nb_pixel_total : 112973 time to create 1 rle with old method : 0.12707090377807617 length of segment : 540 time for calcul the mask position with numpy : 0.0004513263702392578 nb_pixel_total : 13151 time to create 1 rle with old method : 0.015432357788085938 length of segment : 166 time for calcul the mask position with numpy : 0.0002562999725341797 nb_pixel_total : 4908 time to create 1 rle with old method : 0.0059168338775634766 length of segment : 99 time for calcul the mask position with numpy : 0.00023746490478515625 nb_pixel_total : 5935 time to create 1 rle with old method : 0.00708317756652832 length of segment : 85 time for calcul the mask position with numpy : 0.017006874084472656 nb_pixel_total : 899138 time to create 1 rle with new method : 0.07852888107299805 length of segment : 1471 time for calcul the mask position with numpy : 0.0002410411834716797 nb_pixel_total : 10209 time to create 1 rle with old method : 0.011993646621704102 length of segment : 109 time for calcul the mask position with numpy : 0.0017735958099365234 nb_pixel_total : 105678 time to create 1 rle with old method : 0.12678790092468262 length of segment : 598 time for calcul the mask position with numpy : 0.0003039836883544922 nb_pixel_total : 6116 time to create 1 rle with old method : 0.009099960327148438 length of segment : 83 time for calcul the mask position with numpy : 0.0004923343658447266 nb_pixel_total : 19793 time to create 1 rle with old method : 0.0242464542388916 length of segment : 174 time for calcul the mask position with numpy : 0.00038051605224609375 nb_pixel_total : 12118 time to create 1 rle with old method : 0.014267206192016602 length of segment : 136 time spent for convertir_results : 7.654419183731079 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 42 chid ids of type : 3594 Number RLEs to save : 12933 save missing photos in datou_result : time spend for datou_step_exec : 42.2204954624176 time spend to save output : 1.496157169342041 total time spend for step 1 : 43.716652631759644 step2:crop_condition Tue Sep 30 18:01:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 22 ! batch 1 Loaded 42 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3736932 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1759248077_451795 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979691_0ac6d293b86add83643e2992c2a84566_rle_crop_3981191707_0.png', 0, 81, 49, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191717_0.png', 0, 64, 54, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191720_0.png', 0, 76, 98, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191725_0.png', 0, 90, 102, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191729_0.png', 0, 90, 99, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191734_0.png', 0, 103, 81, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248078), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191736_0.png', 0, 127, 136, 0, 1759248078,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 7 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.332923412322998 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 3736932 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1759248081_451795 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386983164_27521a59d13b3e144c82ebe4ae1d945b_rle_crop_3981191695_0.png', 0, 93, 88, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386982938_6808079ecfa646480759471706a382c8_rle_crop_3981191700_0.png', 0, 89, 87, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979801_098c07f7b51f2622eb5d0f70ed156e3f_rle_crop_3981191704_0.png', 0, 346, 551, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e_rle_crop_3981191705_0.png', 0, 94, 118, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61_rle_crop_3981191708_0.png', 0, 75, 77, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191713_0.png', 0, 89, 89, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191722_0.png', 0, 81, 100, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191724_0.png', 0, 95, 83, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191726_0.png', 0, 92, 87, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191730_0.png', 0, 91, 85, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248083), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191733_0.png', 0, 325, 529, 0, 1759248083,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 11 photos in the portfolio 3736932 time of upload the photos Elapsed time : 3.8641858100891113 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! About to upload 22 photos upload in portfolio : 3736932 init cache_photo without model_param we have 22 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1759248105_451795 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436_rle_crop_3981191697_0.png', 0, 966, 952, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436_rle_crop_3981191696_0.png', 0, 334, 565, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa_rle_crop_3981191698_0.png', 0, 242, 185, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386982938_6808079ecfa646480759471706a382c8_rle_crop_3981191701_0.png', 0, 356, 501, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979810_872bb6493e106239a7052452af31cc68_rle_crop_3981191702_0.png', 0, 366, 521, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979804_ba3edf7c1fb25677aedddf404bb8cd1b_rle_crop_3981191703_0.png', 0, 366, 567, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e_rle_crop_3981191706_0.png', 0, 86, 187, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61_rle_crop_3981191709_0.png', 0, 346, 528, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5_rle_crop_3981191710_0.png', 0, 1069, 979, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5_rle_crop_3981191711_0.png', 0, 133, 158, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979673_233851f45638fae7b18b69c1bb899e36_rle_crop_3981191712_0.png', 0, 227, 189, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191715_0.png', 0, 1694, 945, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191716_0.png', 0, 131, 169, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191714_0.png', 0, 225, 121, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191718_0.png', 0, 72, 92, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191719_0.png', 0, 215, 178, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191723_0.png', 0, 327, 519, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191728_0.png', 0, 120, 151, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191727_0.png', 0, 333, 540, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191731_0.png', 0, 1287, 943, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191732_0.png', 0, 129, 109, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248111), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191735_0.png', 0, 160, 166, 0, 1759248111,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 22 photos in the portfolio 3736932 time of upload the photos Elapsed time : 8.00055480003357 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1759248114_451795 INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248114), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa_rle_crop_3981191699_0.png', 0, 95, 79, 0, 1759248114,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! INSERT INTO MTRBack.photos (`timeStamp`, `latitude`, `longitude`, `right_categories`, `tags`, `speed`, `size`, `text`, `altitude`, `width`, `height`, `score`, `created_at`,`source_id`,`place_id`) VALUES (FROM_UNIXTIME(1759248114), 0.0, 0.0, 14, '', 0, 0, '1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191721_0.png', 0, 123, 107, 0, 1759248114,'0',0) batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9499163627624512 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1386983164, 1386982986, 1386982985, 1386982958, 1386982938, 1386979812, 1386979811, 1386979810, 1386979804, 1386979801, 1386979787, 1386979691, 1386979688, 1386979682, 1386979674, 1386979673, 1386979596, 1386979581, 1386979571, 1386979521, 1386979509, 1386979383] Looping around the photos to save general results len do output : 42 /1386995335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995702Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995712Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1386995713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386983164', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982986', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982985', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982958', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982938', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979812', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979811', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979810', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979804', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979801', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979787', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979691', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979688', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979682', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979674', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979673', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979596', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979581', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979571', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979521', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979509', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979383', None, None, None, None, None, '3795462') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 148 time used for this insertion : 0.03984951972961426 save_final save missing photos in datou_result : time spend for datou_step_exec : 39.03427767753601 time spend to save output : 0.04199552536010742 total time spend for step 2 : 39.07627320289612 step3:rle_unique_nms_with_priority Tue Sep 30 18:01:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 42 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.47262048721313477 time for calcul the mask position with numpy : 0.06759786605834961 nb_pixel_total : 2067456 time to create 1 rle with new method : 0.21934103965759277 time for calcul the mask position with numpy : 0.0067331790924072266 nb_pixel_total : 6144 time to create 1 rle with old method : 0.006711721420288086 create new chi : 0.31077098846435547 time to delete rle : 0.10226178169250488 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1256 TO DO : save crop sub photo not yet done ! save time : 0.2829716205596924 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.5135319232940674 time for calcul the mask position with numpy : 0.04355645179748535 nb_pixel_total : 1244348 time to create 1 rle with new method : 0.39977002143859863 time for calcul the mask position with numpy : 0.011077404022216797 nb_pixel_total : 716687 time to create 1 rle with new method : 0.1765148639678955 time for calcul the mask position with numpy : 0.0070192813873291016 nb_pixel_total : 112565 time to create 1 rle with old method : 0.12226319313049316 create new chi : 0.7688369750976562 time to delete rle : 0.0005269050598144531 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 4146 TO DO : save crop sub photo not yet done ! save time : 0.626549482345581 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.04580545425415039 time for calcul the mask position with numpy : 0.01925826072692871 nb_pixel_total : 2034438 time to create 1 rle with new method : 0.09064888954162598 time for calcul the mask position with numpy : 0.005884647369384766 nb_pixel_total : 5774 time to create 1 rle with old method : 0.005890369415283203 time for calcul the mask position with numpy : 0.00623631477355957 nb_pixel_total : 33388 time to create 1 rle with old method : 0.03597116470336914 create new chi : 0.1742556095123291 time to delete rle : 0.0002865791320800781 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1626 TO DO : save crop sub photo not yet done ! save time : 0.3165900707244873 No data in photo_id : 1386982958 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.04162311553955078 time for calcul the mask position with numpy : 0.22446346282958984 nb_pixel_total : 1950210 time to create 1 rle with new method : 0.0898597240447998 time for calcul the mask position with numpy : 0.007269620895385742 nb_pixel_total : 117569 time to create 1 rle with old method : 0.12662458419799805 time for calcul the mask position with numpy : 0.006537914276123047 nb_pixel_total : 5821 time to create 1 rle with old method : 0.00623774528503418 create new chi : 0.47193169593811035 time to delete rle : 0.00035071372985839844 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2264 TO DO : save crop sub photo not yet done ! save time : 0.40129685401916504 No data in photo_id : 1386979812 No data in photo_id : 1386979811 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.0435185432434082 time for calcul the mask position with numpy : 0.03632473945617676 nb_pixel_total : 1954018 time to create 1 rle with new method : 0.4838547706604004 time for calcul the mask position with numpy : 0.012123346328735352 nb_pixel_total : 119582 time to create 1 rle with old method : 0.18159937858581543 create new chi : 0.7244174480438232 time to delete rle : 0.0007641315460205078 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2126 TO DO : save crop sub photo not yet done ! save time : 0.40026092529296875 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.08913064002990723 time for calcul the mask position with numpy : 0.40365004539489746 nb_pixel_total : 1952326 time to create 1 rle with new method : 0.08916902542114258 time for calcul the mask position with numpy : 0.006681203842163086 nb_pixel_total : 121274 time to create 1 rle with old method : 0.13058686256408691 create new chi : 0.6407442092895508 time to delete rle : 0.00034236907958984375 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 2230 TO DO : save crop sub photo not yet done ! save time : 0.3899836540222168 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.0338740348815918 time for calcul the mask position with numpy : 0.2883274555206299 nb_pixel_total : 1967501 time to create 1 rle with new method : 0.3747894763946533 time for calcul the mask position with numpy : 0.0076160430908203125 nb_pixel_total : 106099 time to create 1 rle with old method : 0.11570310592651367 create new chi : 0.7964379787445068 time to delete rle : 0.00039005279541015625 batch 1 Loaded 3 chid ids of type : 3594 ++Number RLEs to save : 2404 TO DO : save crop sub photo not yet done ! save time : 0.3934328556060791 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.1415092945098877 time for calcul the mask position with numpy : 0.24638056755065918 nb_pixel_total : 2056295 time to create 1 rle with new method : 0.41585540771484375 time for calcul the mask position with numpy : 0.010012388229370117 nb_pixel_total : 11116 time to create 1 rle with old method : 0.012270689010620117 time for calcul the mask position with numpy : 0.010041952133178711 nb_pixel_total : 6189 time to create 1 rle with old method : 0.006732940673828125 create new chi : 0.7115278244018555 time to delete rle : 0.0005035400390625 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 1690 TO DO : save crop sub photo not yet done ! save time : 0.35243964195251465 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.04067707061767578 time for calcul the mask position with numpy : 0.2991359233856201 nb_pixel_total : 2071237 time to create 1 rle with new method : 0.09641003608703613 time for calcul the mask position with numpy : 0.006077289581298828 nb_pixel_total : 2363 time to create 1 rle with old method : 0.0027658939361572266 create new chi : 0.41507458686828613 time to delete rle : 0.0002334117889404297 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1178 TO DO : save crop sub photo not yet done ! save time : 0.2879464626312256 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 0.3131415843963623 time for calcul the mask position with numpy : 0.21491074562072754 nb_pixel_total : 1957875 time to create 1 rle with new method : 0.08902740478515625 time for calcul the mask position with numpy : 0.006932973861694336 nb_pixel_total : 111082 time to create 1 rle with old method : 0.11994457244873047 time for calcul the mask position with numpy : 0.006435394287109375 nb_pixel_total : 4643 time to create 1 rle with old method : 0.0051157474517822266 create new chi : 0.45283055305480957 time to delete rle : 0.0003948211669921875 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 2304 TO DO : save crop sub photo not yet done ! save time : 0.42266321182250977 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 0.04781484603881836 time for calcul the mask position with numpy : 0.02060103416442871 nb_pixel_total : 1295125 time to create 1 rle with new method : 0.3705258369445801 time for calcul the mask position with numpy : 0.01065826416015625 nb_pixel_total : 10485 time to create 1 rle with old method : 0.011311769485473633 time for calcul the mask position with numpy : 0.01874518394470215 nb_pixel_total : 767990 time to create 1 rle with new method : 0.19294404983520508 create new chi : 0.6407160758972168 time to delete rle : 0.0004267692565917969 batch 1 Loaded 5 chid ids of type : 3594 ++Number RLEs to save : 3370 TO DO : save crop sub photo not yet done ! save time : 0.51352858543396 No data in photo_id : 1386979674 nb_obj : 1 nb_hashtags : 1 time to prepare the origin masks : 0.0724329948425293 time for calcul the mask position with numpy : 0.07462692260742188 nb_pixel_total : 2053821 time to create 1 rle with new method : 0.26114821434020996 time for calcul the mask position with numpy : 0.006371259689331055 nb_pixel_total : 19779 time to create 1 rle with old method : 0.022024154663085938 create new chi : 0.3734593391418457 time to delete rle : 0.00034236907958984375 batch 1 Loaded 3 chid ids of type : 3594 +Number RLEs to save : 1460 TO DO : save crop sub photo not yet done ! save time : 0.2936580181121826 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 0.3190493583679199 time for calcul the mask position with numpy : 0.013467788696289062 nb_pixel_total : 531320 time to create 1 rle with new method : 0.27931833267211914 time for calcul the mask position with numpy : 0.006151676177978516 nb_pixel_total : 26 time to create 1 rle with old method : 5.841255187988281e-05 time for calcul the mask position with numpy : 0.05042243003845215 nb_pixel_total : 1515741 time to create 1 rle with new method : 0.19954538345336914 time for calcul the mask position with numpy : 0.006457328796386719 nb_pixel_total : 20457 time to create 1 rle with old method : 0.022482633590698242 time for calcul the mask position with numpy : 0.006715536117553711 nb_pixel_total : 6056 time to create 1 rle with old method : 0.006763935089111328 create new chi : 0.6063132286071777 time to delete rle : 0.0005953311920166016 batch 1 Loaded 10 chid ids of type : 3594 ++++++Number RLEs to save : 3655 TO DO : save crop sub photo not yet done ! save time : 0.5457057952880859 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.08345198631286621 time for calcul the mask position with numpy : 0.20560240745544434 nb_pixel_total : 2026865 time to create 1 rle with new method : 0.26019906997680664 time for calcul the mask position with numpy : 0.009209156036376953 nb_pixel_total : 10262 time to create 1 rle with old method : 0.0110321044921875 time for calcul the mask position with numpy : 0.009914875030517578 nb_pixel_total : 4479 time to create 1 rle with old method : 0.004876613616943359 time for calcul the mask position with numpy : 0.009855031967163086 nb_pixel_total : 26670 time to create 1 rle with old method : 0.02828502655029297 time for calcul the mask position with numpy : 0.009752273559570312 nb_pixel_total : 5324 time to create 1 rle with old method : 0.005596160888671875 create new chi : 0.5644199848175049 time to delete rle : 0.00038909912109375 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2032 TO DO : save crop sub photo not yet done ! save time : 0.3880751132965088 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.06499743461608887 time for calcul the mask position with numpy : 0.01881122589111328 nb_pixel_total : 1960928 time to create 1 rle with new method : 0.09303641319274902 time for calcul the mask position with numpy : 0.010381698608398438 nb_pixel_total : 5971 time to create 1 rle with old method : 0.009855508804321289 time for calcul the mask position with numpy : 0.010936498641967773 nb_pixel_total : 101841 time to create 1 rle with old method : 0.11883234977722168 time for calcul the mask position with numpy : 0.005861759185791016 nb_pixel_total : 4860 time to create 1 rle with old method : 0.005179166793823242 create new chi : 0.2851879596710205 time to delete rle : 0.0003979206085205078 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 2484 TO DO : save crop sub photo not yet done ! save time : 0.3895273208618164 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 0.5794713497161865 time for calcul the mask position with numpy : 0.05594801902770996 nb_pixel_total : 1936498 time to create 1 rle with new method : 0.2921257019042969 time for calcul the mask position with numpy : 0.00631260871887207 nb_pixel_total : 13151 time to create 1 rle with old method : 0.014184713363647461 time for calcul the mask position with numpy : 0.006645679473876953 nb_pixel_total : 112768 time to create 1 rle with old method : 0.14680695533752441 time for calcul the mask position with numpy : 0.006002902984619141 nb_pixel_total : 6132 time to create 1 rle with old method : 0.006895542144775391 time for calcul the mask position with numpy : 0.0061872005462646484 nb_pixel_total : 5051 time to create 1 rle with old method : 0.005468845367431641 create new chi : 0.5563220977783203 time to delete rle : 0.0004918575286865234 batch 1 Loaded 9 chid ids of type : 3594 ++++Number RLEs to save : 2855 TO DO : save crop sub photo not yet done ! save time : 0.446871280670166 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 0.08876156806945801 time for calcul the mask position with numpy : 0.18587851524353027 nb_pixel_total : 1047732 time to create 1 rle with new method : 0.08574295043945312 time for calcul the mask position with numpy : 0.006257057189941406 nb_pixel_total : 105678 time to create 1 rle with old method : 0.11259222030639648 time for calcul the mask position with numpy : 0.006123065948486328 nb_pixel_total : 10209 time to create 1 rle with old method : 0.01116180419921875 time for calcul the mask position with numpy : 0.01206660270690918 nb_pixel_total : 899138 time to create 1 rle with new method : 0.27117490768432617 time for calcul the mask position with numpy : 0.006052255630493164 nb_pixel_total : 5935 time to create 1 rle with old method : 0.006399393081665039 time for calcul the mask position with numpy : 0.006299018859863281 nb_pixel_total : 4908 time to create 1 rle with old method : 0.005422353744506836 create new chi : 0.7237720489501953 time to delete rle : 0.0008304119110107422 batch 1 Loaded 11 chid ids of type : 3594 +++++++Number RLEs to save : 5804 TO DO : save crop sub photo not yet done ! save time : 0.7501676082611084 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 0.05777573585510254 time for calcul the mask position with numpy : 0.019474267959594727 nb_pixel_total : 2035573 time to create 1 rle with new method : 0.35161352157592773 time for calcul the mask position with numpy : 0.00616455078125 nb_pixel_total : 12118 time to create 1 rle with old method : 0.013280391693115234 time for calcul the mask position with numpy : 0.006159543991088867 nb_pixel_total : 19793 time to create 1 rle with old method : 0.021191835403442383 time for calcul the mask position with numpy : 0.005680084228515625 nb_pixel_total : 6116 time to create 1 rle with old method : 0.006667613983154297 create new chi : 0.43688082695007324 time to delete rle : 0.0003123283386230469 batch 1 Loaded 7 chid ids of type : 3594 +++Number RLEs to save : 1866 TO DO : save crop sub photo not yet done ! save time : 0.35495495796203613 map_output_result : {1386983164: (0.0, 'Should be the crop_list due to order', 0), 1386982986: (0.0, 'Should be the crop_list due to order', 0), 1386982985: (0.0, 'Should be the crop_list due to order', 0), 1386982958: (0.0, 'Should be the crop_list due to order', 0.0), 1386982938: (0.0, 'Should be the crop_list due to order', 0), 1386979812: (0.0, 'Should be the crop_list due to order', 0.0), 1386979811: (0.0, 'Should be the crop_list due to order', 0.0), 1386979810: (0.0, 'Should be the crop_list due to order', 0), 1386979804: (0.0, 'Should be the crop_list due to order', 0), 1386979801: (0.0, 'Should be the crop_list due to order', 0), 1386979787: (0.0, 'Should be the crop_list due to order', 0), 1386979691: (0.0, 'Should be the crop_list due to order', 0), 1386979688: (0.0, 'Should be the crop_list due to order', 0), 1386979682: (0.0, 'Should be the crop_list due to order', 0), 1386979674: (0.0, 'Should be the crop_list due to order', 0.0), 1386979673: (0.0, 'Should be the crop_list due to order', 0), 1386979596: (0.0, 'Should be the crop_list due to order', 0), 1386979581: (0.0, 'Should be the crop_list due to order', 0), 1386979571: (0.0, 'Should be the crop_list due to order', 0), 1386979521: (0.0, 'Should be the crop_list due to order', 0), 1386979509: (0.0, 'Should be the crop_list due to order', 0), 1386979383: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1386983164, 1386982986, 1386982985, 1386982958, 1386982938, 1386979812, 1386979811, 1386979810, 1386979804, 1386979801, 1386979787, 1386979691, 1386979688, 1386979682, 1386979674, 1386979673, 1386979596, 1386979581, 1386979571, 1386979521, 1386979509, 1386979383] Looping around the photos to save general results len do output : 22 /1386983164.Didn't retrieve data . /1386982986.Didn't retrieve data . /1386982985.Didn't retrieve data . /1386982958.Didn't retrieve data . /1386982938.Didn't retrieve data . /1386979812.Didn't retrieve data . /1386979811.Didn't retrieve data . /1386979810.Didn't retrieve data . /1386979804.Didn't retrieve data . /1386979801.Didn't retrieve data . /1386979787.Didn't retrieve data . /1386979691.Didn't retrieve data . /1386979688.Didn't retrieve data . /1386979682.Didn't retrieve data . /1386979674.Didn't retrieve data . /1386979673.Didn't retrieve data . /1386979596.Didn't retrieve data . /1386979581.Didn't retrieve data . /1386979571.Didn't retrieve data . /1386979521.Didn't retrieve data . /1386979509.Didn't retrieve data . /1386979383.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386983164', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982986', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982985', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982958', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982938', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979812', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979811', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979810', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979804', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979801', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979787', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979691', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979688', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979682', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979674', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979673', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979596', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979581', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979571', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979521', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979509', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979383', None, None, None, None, None, '3795462') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 66 time used for this insertion : 0.03626751899719238 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.130295515060425 time spend to save output : 0.03692364692687988 total time spend for step 3 : 21.167219161987305 step4:ventilate_hashtags_in_portfolio Tue Sep 30 18:02:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 27358921 get user id for portfolio 27358921 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27358921 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','metal','background','pehd','mal_croppe','carton','environnement','pet_clair','autre','pet_fonce','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27358921 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','metal','background','pehd','mal_croppe','carton','environnement','pet_clair','autre','pet_fonce','flou')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27358921 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','metal','background','pehd','mal_croppe','carton','environnement','pet_clair','autre','pet_fonce','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/27359800,27359801,27359802,27359803,27359804,27359805,27359806,27359807,27359808,27359809,27359810?tags=papier,metal,background,pehd,mal_croppe,carton,environnement,pet_clair,autre,pet_fonce,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1386983164, 1386982986, 1386982985, 1386982958, 1386982938, 1386979812, 1386979811, 1386979810, 1386979804, 1386979801, 1386979787, 1386979691, 1386979688, 1386979682, 1386979674, 1386979673, 1386979596, 1386979581, 1386979571, 1386979521, 1386979509, 1386979383] Looping around the photos to save general results len do output : 1 /27358921. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386983164', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982986', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982985', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982958', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982938', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979812', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979811', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979810', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979804', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979801', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979787', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979691', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979688', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979682', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979674', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979673', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979596', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979581', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979571', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979521', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979509', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979383', None, None, None, None, None, '3795462') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 23 time used for this insertion : 0.03704690933227539 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.474543571472168 time spend to save output : 0.03741931915283203 total time spend for step 4 : 5.511962890625 step5:final Tue Sep 30 18:02:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1386983164: ('0.11357573127104378',), 1386982986: ('0.11357573127104378',), 1386982985: ('0.11357573127104378',), 1386982958: ('0.11357573127104378',), 1386982938: ('0.11357573127104378',), 1386979812: ('0.11357573127104378',), 1386979811: ('0.11357573127104378',), 1386979810: ('0.11357573127104378',), 1386979804: ('0.11357573127104378',), 1386979801: ('0.11357573127104378',), 1386979787: ('0.11357573127104378',), 1386979691: ('0.11357573127104378',), 1386979688: ('0.11357573127104378',), 1386979682: ('0.11357573127104378',), 1386979674: ('0.11357573127104378',), 1386979673: ('0.11357573127104378',), 1386979596: ('0.11357573127104378',), 1386979581: ('0.11357573127104378',), 1386979571: ('0.11357573127104378',), 1386979521: ('0.11357573127104378',), 1386979509: ('0.11357573127104378',), 1386979383: ('0.11357573127104378',)} new output for save of step final : {1386983164: ('0.11357573127104378',), 1386982986: ('0.11357573127104378',), 1386982985: ('0.11357573127104378',), 1386982958: ('0.11357573127104378',), 1386982938: ('0.11357573127104378',), 1386979812: ('0.11357573127104378',), 1386979811: ('0.11357573127104378',), 1386979810: ('0.11357573127104378',), 1386979804: ('0.11357573127104378',), 1386979801: ('0.11357573127104378',), 1386979787: ('0.11357573127104378',), 1386979691: ('0.11357573127104378',), 1386979688: ('0.11357573127104378',), 1386979682: ('0.11357573127104378',), 1386979674: ('0.11357573127104378',), 1386979673: ('0.11357573127104378',), 1386979596: ('0.11357573127104378',), 1386979581: ('0.11357573127104378',), 1386979571: ('0.11357573127104378',), 1386979521: ('0.11357573127104378',), 1386979509: ('0.11357573127104378',), 1386979383: ('0.11357573127104378',)} [1386983164, 1386982986, 1386982985, 1386982958, 1386982938, 1386979812, 1386979811, 1386979810, 1386979804, 1386979801, 1386979787, 1386979691, 1386979688, 1386979682, 1386979674, 1386979673, 1386979596, 1386979581, 1386979571, 1386979521, 1386979509, 1386979383] Looping around the photos to save general results len do output : 22 /1386983164.Didn't retrieve data . /1386982986.Didn't retrieve data . /1386982985.Didn't retrieve data . /1386982958.Didn't retrieve data . /1386982938.Didn't retrieve data . /1386979812.Didn't retrieve data . /1386979811.Didn't retrieve data . /1386979810.Didn't retrieve data . /1386979804.Didn't retrieve data . /1386979801.Didn't retrieve data . /1386979787.Didn't retrieve data . /1386979691.Didn't retrieve data . /1386979688.Didn't retrieve data . /1386979682.Didn't retrieve data . /1386979674.Didn't retrieve data . /1386979673.Didn't retrieve data . /1386979596.Didn't retrieve data . /1386979581.Didn't retrieve data . /1386979571.Didn't retrieve data . /1386979521.Didn't retrieve data . /1386979509.Didn't retrieve data . /1386979383.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386983164', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982986', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982985', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982958', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982938', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979812', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979811', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979810', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979804', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979801', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979787', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979691', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979688', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979682', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979674', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979673', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979596', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979581', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979571', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979521', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979509', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979383', None, None, None, None, None, '3795462') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 66 time used for this insertion : 0.08078908920288086 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3948040008544922 time spend to save output : 0.08187246322631836 total time spend for step 5 : 0.47667646408081055 step6:blur_detection Tue Sep 30 18:02:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1759248029_451795_1386983164_27521a59d13b3e144c82ebe4ae1d945b.jpg resize: (1080, 1920) 1386983164 -2.4482536795326157 treat image : temp/1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436.jpg resize: (1080, 1920) 1386982986 -0.46079424691313614 treat image : temp/1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa.jpg resize: (1080, 1920) 1386982985 -2.1998428215133967 treat image : temp/1759248029_451795_1386982958_edcba36af5cd8ba93f657c4b6a34f008.jpg resize: (1080, 1920) 1386982958 -1.389330523795732 treat image : temp/1759248029_451795_1386982938_6808079ecfa646480759471706a382c8.jpg resize: (1080, 1920) 1386982938 -1.8687193704304004 treat image : temp/1759248029_451795_1386979812_8c7a6e9ffeb73b25ce999723f2cc8751.jpg resize: (1080, 1920) 1386979812 0.5292931165626452 treat image : temp/1759248029_451795_1386979811_128ddb53a01dabe322539a354a4972a9.jpg resize: (1080, 1920) 1386979811 -1.8654206080521887 treat image : temp/1759248029_451795_1386979810_872bb6493e106239a7052452af31cc68.jpg resize: (1080, 1920) 1386979810 -4.269531693523343 treat image : temp/1759248029_451795_1386979804_ba3edf7c1fb25677aedddf404bb8cd1b.jpg resize: (1080, 1920) 1386979804 -0.2215471723523826 treat image : temp/1759248029_451795_1386979801_098c07f7b51f2622eb5d0f70ed156e3f.jpg resize: (1080, 1920) 1386979801 -2.929187936612265 treat image : temp/1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e.jpg resize: (1080, 1920) 1386979787 -2.0505896134607204 treat image : temp/1759248029_451795_1386979691_0ac6d293b86add83643e2992c2a84566.jpg resize: (1080, 1920) 1386979691 -2.6818603008912176 treat image : temp/1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61.jpg resize: (1080, 1920) 1386979688 -2.541929721736901 treat image : temp/1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5.jpg resize: (1080, 1920) 1386979682 -0.7900053906016516 treat image : temp/1759248029_451795_1386979674_7297123e291d2653c26b0dac8c864081.jpg resize: (1080, 1920) 1386979674 -1.0801844565132503 treat image : temp/1759248029_451795_1386979673_233851f45638fae7b18b69c1bb899e36.jpg resize: (1080, 1920) 1386979673 -2.0677334192815175 treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934.jpg resize: (1080, 1920) 1386979596 -2.007540859299859 treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f.jpg resize: (1080, 1920) 1386979581 -2.2467412010489767 treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84.jpg resize: (1080, 1920) 1386979571 -0.5048623585853665 treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161.jpg resize: (1080, 1920) 1386979521 -3.1984140302275224 treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a.jpg resize: (1080, 1920) 1386979509 -1.8403465291791579 treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45.jpg resize: (1080, 1920) 1386979383 -2.2763926148472162 treat image : temp/1759248029_451795_1386979691_0ac6d293b86add83643e2992c2a84566_rle_crop_3981191707_0.png resize: (49, 81) 1386995335 -2.490074977391412 treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191717_0.png resize: (54, 64) 1386995349 -0.5461311676242999 treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191720_0.png resize: (98, 76) 1386995361 -1.8854333001871235 treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191725_0.png resize: (102, 90) 1386995378 -1.8947841386377924 treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191729_0.png resize: (99, 90) 1386995398 -1.878709273104884 treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191734_0.png resize: (81, 103) 1386995403 -0.5786550343832325 treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191736_0.png resize: (136, 127) 1386995404 -1.0625824228934149 treat image : temp/1759248029_451795_1386983164_27521a59d13b3e144c82ebe4ae1d945b_rle_crop_3981191695_0.png resize: (88, 93) 1386995408 -0.40675391709006886 treat image : temp/1759248029_451795_1386982938_6808079ecfa646480759471706a382c8_rle_crop_3981191700_0.png resize: (87, 89) 1386995409 -0.1849604674227747 treat image : temp/1759248029_451795_1386979801_098c07f7b51f2622eb5d0f70ed156e3f_rle_crop_3981191704_0.png resize: (551, 346) 1386995411 -0.6141388399619281 treat image : temp/1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e_rle_crop_3981191705_0.png resize: (118, 94) 1386995412 -2.159747779644811 treat image : temp/1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61_rle_crop_3981191708_0.png resize: (77, 75) 1386995413 2.3220425907228455 treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191713_0.png resize: (89, 89) 1386995414 -0.06303873109927531 treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191722_0.png resize: (100, 81) 1386995415 -1.8053574405973316 treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191724_0.png resize: (83, 95) 1386995416 -0.35385169784245457 treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191726_0.png resize: (87, 92) 1386995417 -0.1866637575167468 treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191730_0.png resize: (85, 91) 1386995418 -0.20667096582363703 treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191733_0.png resize: (529, 325) 1386995419 -0.3369020789743006 treat image : temp/1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436_rle_crop_3981191697_0.png resize: (952, 966) 1386995686 0.33219837656438944 treat image : temp/1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436_rle_crop_3981191696_0.png resize: (565, 334) 1386995687 0.21784686054694188 treat image : temp/1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa_rle_crop_3981191698_0.png resize: (185, 242) 1386995688 -1.3700231879367475 treat image : temp/1759248029_451795_1386982938_6808079ecfa646480759471706a382c8_rle_crop_3981191701_0.png resize: (501, 356) 1386995689 -0.1450666045742308 treat image : temp/1759248029_451795_1386979810_872bb6493e106239a7052452af31cc68_rle_crop_3981191702_0.png resize: (521, 366) 1386995690 0.15840233799402148 treat image : temp/1759248029_451795_1386979804_ba3edf7c1fb25677aedddf404bb8cd1b_rle_crop_3981191703_0.png resize: (567, 366) 1386995693 0.29584687220154343 treat image : temp/1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e_rle_crop_3981191706_0.png resize: (187, 86) 1386995694 -1.9162531367338969 treat image : temp/1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61_rle_crop_3981191709_0.png resize: (528, 346) 1386995695 -0.25337514765729485 treat image : temp/1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5_rle_crop_3981191710_0.png resize: (979, 1069) 1386995696 0.25457902332979315 treat image : temp/1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5_rle_crop_3981191711_0.png resize: (158, 133) 1386995697 -2.158688542941417 treat image : temp/1759248029_451795_1386979673_233851f45638fae7b18b69c1bb899e36_rle_crop_3981191712_0.png resize: (189, 227) 1386995698 -3.325740504469249 treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191715_0.png resize: (945, 1694) 1386995699 -2.2173240567510146 treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191716_0.png resize: (169, 131) 1386995701 -0.8452614225503354 treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191714_0.png resize: (121, 225) 1386995702 -2.293929177455 treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191718_0.png resize: (92, 72) 1386995703 -4.597004064186332 treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191719_0.png resize: (178, 215) 1386995704 -0.7276494223505724 treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191723_0.png resize: (519, 327) 1386995705 -0.011144730203447167 treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191728_0.png resize: (151, 120) 1386995706 -4.479793554860878 treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191727_0.png resize: (540, 333) 1386995707 0.008083158673489724 treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191731_0.png resize: (943, 1287) 1386995708 -1.1441837299641486 treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191732_0.png resize: (109, 129) 1386995709 -3.104461269455546 treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191735_0.png resize: (166, 160) 1386995710 -3.550662492159202 treat image : temp/1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa_rle_crop_3981191699_0.png resize: (79, 95) 1386995712 -0.7813519217309145 treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191721_0.png resize: (107, 123) 1386995713 0.7232337020945533 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 64 time used for this insertion : 0.03761482238769531 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 64 time used for this insertion : 0.03820347785949707 save missing photos in datou_result : time spend for datou_step_exec : 20.360270023345947 time spend to save output : 0.0928337574005127 total time spend for step 6 : 20.45310378074646 step7:brightness Tue Sep 30 18:02:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1759248029_451795_1386983164_27521a59d13b3e144c82ebe4ae1d945b.jpg treat image : temp/1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436.jpg treat image : temp/1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa.jpg treat image : temp/1759248029_451795_1386982958_edcba36af5cd8ba93f657c4b6a34f008.jpg treat image : temp/1759248029_451795_1386982938_6808079ecfa646480759471706a382c8.jpg treat image : temp/1759248029_451795_1386979812_8c7a6e9ffeb73b25ce999723f2cc8751.jpg treat image : temp/1759248029_451795_1386979811_128ddb53a01dabe322539a354a4972a9.jpg treat image : temp/1759248029_451795_1386979810_872bb6493e106239a7052452af31cc68.jpg treat image : temp/1759248029_451795_1386979804_ba3edf7c1fb25677aedddf404bb8cd1b.jpg treat image : temp/1759248029_451795_1386979801_098c07f7b51f2622eb5d0f70ed156e3f.jpg treat image : temp/1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e.jpg treat image : temp/1759248029_451795_1386979691_0ac6d293b86add83643e2992c2a84566.jpg treat image : temp/1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61.jpg treat image : temp/1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5.jpg treat image : temp/1759248029_451795_1386979674_7297123e291d2653c26b0dac8c864081.jpg treat image : temp/1759248029_451795_1386979673_233851f45638fae7b18b69c1bb899e36.jpg treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934.jpg treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f.jpg treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84.jpg treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161.jpg treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a.jpg treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45.jpg treat image : temp/1759248029_451795_1386979691_0ac6d293b86add83643e2992c2a84566_rle_crop_3981191707_0.png treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191717_0.png treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191720_0.png treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191725_0.png treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191729_0.png treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191734_0.png treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191736_0.png treat image : temp/1759248029_451795_1386983164_27521a59d13b3e144c82ebe4ae1d945b_rle_crop_3981191695_0.png treat image : temp/1759248029_451795_1386982938_6808079ecfa646480759471706a382c8_rle_crop_3981191700_0.png treat image : temp/1759248029_451795_1386979801_098c07f7b51f2622eb5d0f70ed156e3f_rle_crop_3981191704_0.png treat image : temp/1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e_rle_crop_3981191705_0.png treat image : temp/1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61_rle_crop_3981191708_0.png treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191713_0.png treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191722_0.png treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191724_0.png treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191726_0.png treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191730_0.png treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191733_0.png treat image : temp/1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436_rle_crop_3981191697_0.png treat image : temp/1759248029_451795_1386982986_9d6d7982c21e7790a3b93cbaa8dfa436_rle_crop_3981191696_0.png treat image : temp/1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa_rle_crop_3981191698_0.png treat image : temp/1759248029_451795_1386982938_6808079ecfa646480759471706a382c8_rle_crop_3981191701_0.png treat image : temp/1759248029_451795_1386979810_872bb6493e106239a7052452af31cc68_rle_crop_3981191702_0.png treat image : temp/1759248029_451795_1386979804_ba3edf7c1fb25677aedddf404bb8cd1b_rle_crop_3981191703_0.png treat image : temp/1759248029_451795_1386979787_f87a4a7bf86624dd752126960158ca7e_rle_crop_3981191706_0.png treat image : temp/1759248029_451795_1386979688_cbf2c03978a76fed709f278be6983a61_rle_crop_3981191709_0.png treat image : temp/1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5_rle_crop_3981191710_0.png treat image : temp/1759248029_451795_1386979682_a2b95dd7a388773aeec4960d2b6fbfe5_rle_crop_3981191711_0.png treat image : temp/1759248029_451795_1386979673_233851f45638fae7b18b69c1bb899e36_rle_crop_3981191712_0.png treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191715_0.png treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191716_0.png treat image : temp/1759248029_451795_1386979596_23547aeaae0b616eb310961a7655e934_rle_crop_3981191714_0.png treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191718_0.png treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191719_0.png treat image : temp/1759248029_451795_1386979571_5a2ee636c8a40991bb5de4a3aa0d7b84_rle_crop_3981191723_0.png treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191728_0.png treat image : temp/1759248029_451795_1386979521_94cf5b848e34e986f7182ee3e1cb6161_rle_crop_3981191727_0.png treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191731_0.png treat image : temp/1759248029_451795_1386979509_4314b45619e7ee07fe0d9a249047415a_rle_crop_3981191732_0.png treat image : temp/1759248029_451795_1386979383_ad85a30b9a5e2cda1ca73dc2c7006b45_rle_crop_3981191735_0.png treat image : temp/1759248029_451795_1386982985_b08bcde0bf67dce50ec2d51798f97bfa_rle_crop_3981191699_0.png treat image : temp/1759248029_451795_1386979581_81a3e200f04dc23d94b5687c18a8e22f_rle_crop_3981191721_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 64 time used for this insertion : 0.11063313484191895 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 64 time used for this insertion : 0.03939104080200195 save missing photos in datou_result : time spend for datou_step_exec : 5.731298923492432 time spend to save output : 0.16881513595581055 total time spend for step 7 : 5.900114059448242 step8:velours_tree Tue Sep 30 18:02:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2104945182800293 time spend to save output : 5.507469177246094e-05 total time spend for step 8 : 0.21054959297180176 step9:send_mail_cod Tue Sep 30 18:02:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P27358921_30-09-2025_18_02_48.pdf 27359800 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette273598001759248168 27359801 imagette273598011759248169 27359802 imagette273598021759248169 27359803 imagette273598031759248169 27359804 imagette273598041759248169 27359805 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette273598051759248169 27359807 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette273598071759248170 27359808 change filename to text .change filename to text .imagette273598081759248172 27359809 imagette273598091759248172 27359810 imagette273598101759248172 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=27358921 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/27359800,27359801,27359802,27359803,27359804,27359805,27359806,27359807,27359808,27359809,27359810?tags=papier,metal,background,pehd,mal_croppe,carton,environnement,pet_clair,autre,pet_fonce,flou args[1386983164] : ((1386983164, -2.4482536795326157, 492609224), (1386983164, 0.4907398521691151, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386982986] : ((1386982986, -0.46079424691313614, 492688767), (1386982986, 0.5293399987314071, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386982985] : ((1386982985, -2.1998428215133967, 492609224), (1386982985, 0.4331516413842789, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386982958] : ((1386982958, -1.389330523795732, 492688767), (1386982958, 0.4871911826182629, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386982938] : ((1386982938, -1.8687193704304004, 492688767), (1386982938, 0.5996830914087936, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979812] : ((1386979812, 0.5292931165626452, 492688767), (1386979812, 0.5109410534018727, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979811] : ((1386979811, -1.8654206080521887, 492688767), (1386979811, 0.554311206080565, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979810] : ((1386979810, -4.269531693523343, 492609224), (1386979810, 0.4815870686545229, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979804] : ((1386979804, -0.2215471723523826, 492688767), (1386979804, 0.5750291140230477, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979801] : ((1386979801, -2.929187936612265, 492609224), (1386979801, 0.47248504028422045, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979787] : ((1386979787, -2.0505896134607204, 492609224), (1386979787, 0.6616379990236109, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979691] : ((1386979691, -2.6818603008912176, 492609224), (1386979691, 0.7498461591832016, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979688] : ((1386979688, -2.541929721736901, 492609224), (1386979688, 0.6263224642072392, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979682] : ((1386979682, -0.7900053906016516, 492688767), (1386979682, 0.6203431982241064, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979674] : ((1386979674, -1.0801844565132503, 492688767), (1386979674, 0.6758375896783072, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979673] : ((1386979673, -2.0677334192815175, 492609224), (1386979673, 0.5454771339641837, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979596] : ((1386979596, -2.007540859299859, 492609224), (1386979596, 0.3463125649942825, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979581] : ((1386979581, -2.2467412010489767, 492609224), (1386979581, 0.3728197981155471, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979571] : ((1386979571, -0.5048623585853665, 492688767), (1386979571, 0.31014319219646874, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979521] : ((1386979521, -3.1984140302275224, 492609224), (1386979521, 0.3493653086166173, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979509] : ((1386979509, -1.8403465291791579, 492688767), (1386979509, 0.29565607948823386, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com args[1386979383] : ((1386979383, -2.2763926148472162, 492609224), (1386979383, 0.6570242056018627, 2107752395), '0.11357573127104378') We are sending mail with results at report@fotonower.com refus_total : 0.11357573127104378 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos_view ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27358921 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358921_30-09-2025_18_02_48.pdf results_Auto_P27358921_30-09-2025_18_02_48.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358921_30-09-2025_18_02_48.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','27358921','results_Auto_P27358921_30-09-2025_18_02_48.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358921_30-09-2025_18_02_48.pdf','pdf','','0.66','0.11357573127104378') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/27358921

https://www.fotonower.com/image?json=false&list_photos_id=1386983164
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386982986
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386982985
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386982958
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386982938
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979812
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979811
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979810
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979804
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979801
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979787
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979691
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979688
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979682
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979674
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979673
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979596
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979581
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979571
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979521
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979509
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1386979383
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 11.36%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/27359800?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/27359805?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/27359807?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/27359808?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358921_30-09-2025_18_02_48.pdf.

Lien vers velours :https://marlene.fotonower.com/velours/27359800,27359801,27359802,27359803,27359804,27359805,27359806,27359807,27359808,27359809,27359810?tags=papier,metal,background,pehd,mal_croppe,carton,environnement,pet_clair,autre,pet_fonce,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Tue, 30 Sep 2025 16:02:55 GMT Content-Length: 0 Connection: close X-Message-Id: 6aOuOIocQoyTIEDL3mGU6g Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1386983164, 1386982986, 1386982985, 1386982958, 1386982938, 1386979812, 1386979811, 1386979810, 1386979804, 1386979801, 1386979787, 1386979691, 1386979688, 1386979682, 1386979674, 1386979673, 1386979596, 1386979581, 1386979571, 1386979521, 1386979509, 1386979383] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386983164', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982986', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982985', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982958', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982938', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979812', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979811', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979810', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979804', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979801', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979787', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979691', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979688', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979682', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979674', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979673', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979596', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979581', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979571', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979521', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979509', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979383', None, None, None, None, None, '3795462') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 22 time used for this insertion : 0.03777766227722168 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.755012035369873 time spend to save output : 0.038068532943725586 total time spend for step 9 : 6.793080568313599 step10:split_time_score Tue Sep 30 18:02:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('16', 22),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 30092025 27358921 Nombre de photos uploadées : 22 / 23040 (0%) 30092025 27358921 Nombre de photos taguées (types de déchets): 0 / 22 (0%) 30092025 27358921 Nombre de photos taguées (volume) : 0 / 22 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 1.33514404296875e-05 ?????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.000997781753540039 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.6578695774078369 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.25469097222222226 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27344240_30-09-2025_17_08_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27344240 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27344240 AND mptpi.`type`=3594 To do Qualite : 0.1986372843013468 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27344242_30-09-2025_17_08_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27344242 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27344242 AND mptpi.`type`=3594 To do Qualite : 0.040582133058984914 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27344246_30-09-2025_17_08_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27344246 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27344246 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27344248 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27344254 order by id desc limit 1 Qualite : 0.02596068029835391 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27355641_30-09-2025_17_22_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27355641 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27355641 AND mptpi.`type`=3594 To do Qualite : 0.025566601447677834 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27355643_30-09-2025_17_12_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27355643 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27355643 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358906 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358912 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358916 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358918 order by id desc limit 1 Qualite : 0.11357573127104378 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358921_30-09-2025_18_02_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358921 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27358921 AND mptpi.`type`=3594 To do Qualite : 0.1420479145233196 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358922_30-09-2025_17_41_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358922 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27358922 AND mptpi.`type`=3594 To do Qualite : 0.07097040037722906 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P27358924_30-09-2025_17_51_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 27358924 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=27358924 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'30092025': {'nb_upload': 22, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1386983164, 1386982986, 1386982985, 1386982958, 1386982938, 1386979812, 1386979811, 1386979810, 1386979804, 1386979801, 1386979787, 1386979691, 1386979688, 1386979682, 1386979674, 1386979673, 1386979596, 1386979581, 1386979571, 1386979521, 1386979509, 1386979383] Looping around the photos to save general results len do output : 1 /27358921Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386983164', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982986', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982985', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982958', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386982938', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979812', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979811', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979810', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979804', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979801', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979787', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979691', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979688', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979682', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979674', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979673', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979596', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979581', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979571', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979521', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979509', None, None, None, None, None, '3795462') ('3318', None, None, None, None, None, None, None, '3795462') ('3318', '27358921', '1386979383', None, None, None, None, None, '3795462') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 23 time used for this insertion : 0.04024696350097656 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.659799337387085 time spend to save output : 0.040537357330322266 total time spend for step 10 : 6.700336694717407 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 22 set_done_treatment 76.26user 38.67system 2:37.66elapsed 72%CPU (0avgtext+0avgdata 2250024maxresident)k 459248inputs+45520outputs (12854major+2394374minor)pagefaults 0swaps