python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2294421 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2983358'] with mtr_portfolio_ids : ['23439680'] and first list_photo_ids : [] new path : /proc/2294421/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.2399160861968994 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu May 29 13:10:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4363 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-29 13:10:31.728892: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-29 13:10:31.755255: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-29 13:10:31.756807: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff330000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-29 13:10:31.756864: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-29 13:10:31.759589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-29 13:10:31.908815: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3b1c82c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-29 13:10:31.908863: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-29 13:10:31.909791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-29 13:10:31.910153: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-29 13:10:31.913791: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-29 13:10:31.916221: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-29 13:10:31.916607: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-29 13:10:31.919331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-29 13:10:31.920827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-29 13:10:31.926428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-29 13:10:31.927841: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-29 13:10:31.927943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-29 13:10:31.928684: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-29 13:10:31.928703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-29 13:10:31.928714: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-29 13:10:31.929986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3758 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-29 13:10:32.201705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-29 13:10:32.201832: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-29 13:10:32.201854: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-29 13:10:32.201871: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-29 13:10:32.201887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-29 13:10:32.201903: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-29 13:10:32.201918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-29 13:10:32.201935: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-29 13:10:32.202824: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-29 13:10:32.203894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-29 13:10:32.203938: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-29 13:10:32.203959: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-29 13:10:32.203981: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-29 13:10:32.203997: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-29 13:10:32.204015: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-29 13:10:32.204032: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-29 13:10:32.204051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-29 13:10:32.204959: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-29 13:10:32.204991: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-29 13:10:32.205001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-29 13:10:32.205009: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-29 13:10:32.205961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3758 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-29 13:10:41.512052: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-29 13:10:41.710018: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-29 13:10:41.826197: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-05-29 13:10:41.840112: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-05-29 13:10:43.063443: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-05-29 13:10:43.065140: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2025-05-29 13:10:46.330391: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.17G (3405643776 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-29 13:10:47.126617: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-29 13:10:47.126661: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 15.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 error in detect the image : temp/1748517027_2294421_1361620898_2982070e5c0bc8af62e710c2bd9ceba3.jpg 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] [[mrcnn_detection/ExpandDims_2/_84]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] 0 successful operations. 0 derived errors ignored. [Op:__inference_keras_scratch_graph_13584] Function call stack: keras_scratch_graph -> keras_scratch_graph NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.07972 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 error in detect the image : temp/1748517027_2294421_1361618422_c3b41996025eb29beeb26e6536aed7f7.jpg 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] [[mrcnn_detection/ExpandDims_2/_84]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node conv1/convolution (defined at /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3007) ]] 0 successful operations. 0 derived errors ignored. [Op:__inference_keras_scratch_graph_13584] Function call stack: keras_scratch_graph -> keras_scratch_graph NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 25.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 38 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 38.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 29.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 37 Detection mask done ! Trying to reset tf kernel 2295053 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 214 tf kernel not reseted sub process len(results) : 6 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 6 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4363 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0009229183197021484 nb_pixel_total : 34402 time to create 1 rle with old method : 0.038750410079956055 length of segment : 247 time for calcul the mask position with numpy : 0.0020694732666015625 nb_pixel_total : 86877 time to create 1 rle with old method : 0.09681200981140137 length of segment : 464 time for calcul the mask position with numpy : 0.0006458759307861328 nb_pixel_total : 41204 time to create 1 rle with old method : 0.04740309715270996 length of segment : 167 time for calcul the mask position with numpy : 0.0016367435455322266 nb_pixel_total : 86514 time to create 1 rle with old method : 0.09711003303527832 length of segment : 375 time for calcul the mask position with numpy : 0.014653444290161133 nb_pixel_total : 289861 time to create 1 rle with new method : 0.037868499755859375 length of segment : 481 time for calcul the mask position with numpy : 0.0024099349975585938 nb_pixel_total : 167876 time to create 1 rle with new method : 0.006684780120849609 length of segment : 502 time for calcul the mask position with numpy : 0.0012822151184082031 nb_pixel_total : 45868 time to create 1 rle with old method : 0.05269789695739746 length of segment : 847 time for calcul the mask position with numpy : 0.0005662441253662109 nb_pixel_total : 16160 time to create 1 rle with old method : 0.01795816421508789 length of segment : 157 time for calcul the mask position with numpy : 0.0005419254302978516 nb_pixel_total : 24325 time to create 1 rle with old method : 0.026629209518432617 length of segment : 325 time for calcul the mask position with numpy : 0.0019741058349609375 nb_pixel_total : 131681 time to create 1 rle with old method : 0.1464388370513916 length of segment : 426 time for calcul the mask position with numpy : 0.0017292499542236328 nb_pixel_total : 94121 time to create 1 rle with old method : 0.09702849388122559 length of segment : 363 time for calcul the mask position with numpy : 0.0027985572814941406 nb_pixel_total : 122152 time to create 1 rle with old method : 0.13013696670532227 length of segment : 449 time for calcul the mask position with numpy : 0.0007379055023193359 nb_pixel_total : 43627 time to create 1 rle with old method : 0.04559183120727539 length of segment : 249 time for calcul the mask position with numpy : 0.00041794776916503906 nb_pixel_total : 18319 time to create 1 rle with old method : 0.020513057708740234 length of segment : 221 time for calcul the mask position with numpy : 0.0006253719329833984 nb_pixel_total : 35658 time to create 1 rle with old method : 0.038060903549194336 length of segment : 233 time for calcul the mask position with numpy : 0.00030732154846191406 nb_pixel_total : 15945 time to create 1 rle with old method : 0.017453432083129883 length of segment : 146 time for calcul the mask position with numpy : 0.00048828125 nb_pixel_total : 29892 time to create 1 rle with old method : 0.0330510139465332 length of segment : 198 time for calcul the mask position with numpy : 0.00019288063049316406 nb_pixel_total : 9873 time to create 1 rle with old method : 0.011203527450561523 length of segment : 119 time for calcul the mask position with numpy : 0.0012373924255371094 nb_pixel_total : 76136 time to create 1 rle with old method : 0.07974767684936523 length of segment : 438 time for calcul the mask position with numpy : 0.0008246898651123047 nb_pixel_total : 59838 time to create 1 rle with old method : 0.06282758712768555 length of segment : 340 time for calcul the mask position with numpy : 0.00028324127197265625 nb_pixel_total : 10915 time to create 1 rle with old method : 0.011725902557373047 length of segment : 120 time for calcul the mask position with numpy : 0.0008718967437744141 nb_pixel_total : 45658 time to create 1 rle with old method : 0.04880380630493164 length of segment : 229 time for calcul the mask position with numpy : 0.0006172657012939453 nb_pixel_total : 32913 time to create 1 rle with old method : 0.03949117660522461 length of segment : 156 time for calcul the mask position with numpy : 0.0003352165222167969 nb_pixel_total : 15828 time to create 1 rle with old method : 0.016878843307495117 length of segment : 169 time for calcul the mask position with numpy : 0.0006394386291503906 nb_pixel_total : 26935 time to create 1 rle with old method : 0.02905893325805664 length of segment : 252 time for calcul the mask position with numpy : 0.0009691715240478516 nb_pixel_total : 47912 time to create 1 rle with old method : 0.05048871040344238 length of segment : 301 time for calcul the mask position with numpy : 0.0004754066467285156 nb_pixel_total : 27027 time to create 1 rle with old method : 0.028729915618896484 length of segment : 200 time for calcul the mask position with numpy : 0.001489877700805664 nb_pixel_total : 76689 time to create 1 rle with old method : 0.07985424995422363 length of segment : 452 time for calcul the mask position with numpy : 0.0016422271728515625 nb_pixel_total : 103497 time to create 1 rle with old method : 0.10781621932983398 length of segment : 332 time for calcul the mask position with numpy : 0.0013883113861083984 nb_pixel_total : 61153 time to create 1 rle with old method : 0.06284666061401367 length of segment : 287 time for calcul the mask position with numpy : 0.0013492107391357422 nb_pixel_total : 76949 time to create 1 rle with old method : 0.08198237419128418 length of segment : 328 time for calcul the mask position with numpy : 0.0005159378051757812 nb_pixel_total : 30490 time to create 1 rle with old method : 0.031998634338378906 length of segment : 201 time for calcul the mask position with numpy : 0.0004189014434814453 nb_pixel_total : 21004 time to create 1 rle with old method : 0.02256488800048828 length of segment : 139 time for calcul the mask position with numpy : 0.003438711166381836 nb_pixel_total : 126193 time to create 1 rle with old method : 0.13102245330810547 length of segment : 477 time for calcul the mask position with numpy : 0.0006909370422363281 nb_pixel_total : 14602 time to create 1 rle with old method : 0.01562666893005371 length of segment : 143 time for calcul the mask position with numpy : 0.000762939453125 nb_pixel_total : 36268 time to create 1 rle with old method : 0.038387298583984375 length of segment : 190 time for calcul the mask position with numpy : 0.0017139911651611328 nb_pixel_total : 43380 time to create 1 rle with old method : 0.0460050106048584 length of segment : 227 time for calcul the mask position with numpy : 0.0012695789337158203 nb_pixel_total : 26335 time to create 1 rle with old method : 0.02773118019104004 length of segment : 244 time for calcul the mask position with numpy : 0.0005803108215332031 nb_pixel_total : 14804 time to create 1 rle with old method : 0.015949487686157227 length of segment : 118 time for calcul the mask position with numpy : 0.00019621849060058594 nb_pixel_total : 4563 time to create 1 rle with old method : 0.00490880012512207 length of segment : 125 time for calcul the mask position with numpy : 0.001959562301635742 nb_pixel_total : 50559 time to create 1 rle with old method : 0.053163766860961914 length of segment : 223 time for calcul the mask position with numpy : 0.007215738296508789 nb_pixel_total : 164003 time to create 1 rle with new method : 0.014156579971313477 length of segment : 638 time for calcul the mask position with numpy : 0.0021898746490478516 nb_pixel_total : 49455 time to create 1 rle with old method : 0.05195164680480957 length of segment : 392 time for calcul the mask position with numpy : 0.0012984275817871094 nb_pixel_total : 26343 time to create 1 rle with old method : 0.028231143951416016 length of segment : 183 time for calcul the mask position with numpy : 0.0007450580596923828 nb_pixel_total : 25537 time to create 1 rle with old method : 0.02778768539428711 length of segment : 200 time for calcul the mask position with numpy : 0.0013816356658935547 nb_pixel_total : 25271 time to create 1 rle with old method : 0.027869462966918945 length of segment : 312 time for calcul the mask position with numpy : 0.0012679100036621094 nb_pixel_total : 25943 time to create 1 rle with old method : 0.02871990203857422 length of segment : 221 time for calcul the mask position with numpy : 0.00054931640625 nb_pixel_total : 12399 time to create 1 rle with old method : 0.013826847076416016 length of segment : 116 time for calcul the mask position with numpy : 0.0016322135925292969 nb_pixel_total : 50194 time to create 1 rle with old method : 0.05491185188293457 length of segment : 258 time for calcul the mask position with numpy : 0.0005512237548828125 nb_pixel_total : 16472 time to create 1 rle with old method : 0.018785953521728516 length of segment : 148 time for calcul the mask position with numpy : 0.0004177093505859375 nb_pixel_total : 8185 time to create 1 rle with old method : 0.009380340576171875 length of segment : 72 time for calcul the mask position with numpy : 0.0015039443969726562 nb_pixel_total : 19965 time to create 1 rle with old method : 0.021968364715576172 length of segment : 296 time for calcul the mask position with numpy : 0.0005125999450683594 nb_pixel_total : 10668 time to create 1 rle with old method : 0.01182103157043457 length of segment : 93 time for calcul the mask position with numpy : 0.008024930953979492 nb_pixel_total : 197790 time to create 1 rle with new method : 0.015448331832885742 length of segment : 655 time for calcul the mask position with numpy : 0.0012173652648925781 nb_pixel_total : 39630 time to create 1 rle with old method : 0.04348611831665039 length of segment : 297 time for calcul the mask position with numpy : 0.008107423782348633 nb_pixel_total : 104233 time to create 1 rle with old method : 0.11398577690124512 length of segment : 414 time for calcul the mask position with numpy : 0.0005495548248291016 nb_pixel_total : 14861 time to create 1 rle with old method : 0.016798973083496094 length of segment : 132 time for calcul the mask position with numpy : 0.0013587474822998047 nb_pixel_total : 26265 time to create 1 rle with old method : 0.029686927795410156 length of segment : 207 time for calcul the mask position with numpy : 0.0017809867858886719 nb_pixel_total : 68149 time to create 1 rle with old method : 0.07296347618103027 length of segment : 231 time for calcul the mask position with numpy : 0.0031006336212158203 nb_pixel_total : 154784 time to create 1 rle with new method : 0.005469083786010742 length of segment : 280 time for calcul the mask position with numpy : 0.0013234615325927734 nb_pixel_total : 29208 time to create 1 rle with old method : 0.03170061111450195 length of segment : 226 time for calcul the mask position with numpy : 0.0028455257415771484 nb_pixel_total : 92662 time to create 1 rle with old method : 0.1014091968536377 length of segment : 397 time for calcul the mask position with numpy : 0.0072057247161865234 nb_pixel_total : 139234 time to create 1 rle with old method : 0.14740705490112305 length of segment : 886 time for calcul the mask position with numpy : 0.0012328624725341797 nb_pixel_total : 43892 time to create 1 rle with old method : 0.04562187194824219 length of segment : 247 time for calcul the mask position with numpy : 0.002156972885131836 nb_pixel_total : 62391 time to create 1 rle with old method : 0.07010912895202637 length of segment : 421 time for calcul the mask position with numpy : 0.001005411148071289 nb_pixel_total : 35029 time to create 1 rle with old method : 0.03796267509460449 length of segment : 209 time for calcul the mask position with numpy : 0.008278369903564453 nb_pixel_total : 316704 time to create 1 rle with new method : 0.02740335464477539 length of segment : 881 time for calcul the mask position with numpy : 0.0009307861328125 nb_pixel_total : 41135 time to create 1 rle with old method : 0.04636955261230469 length of segment : 234 time for calcul the mask position with numpy : 0.0024709701538085938 nb_pixel_total : 168428 time to create 1 rle with new method : 0.006369352340698242 length of segment : 292 time for calcul the mask position with numpy : 0.0019979476928710938 nb_pixel_total : 61250 time to create 1 rle with old method : 0.06741118431091309 length of segment : 233 time for calcul the mask position with numpy : 0.0008184909820556641 nb_pixel_total : 24346 time to create 1 rle with old method : 0.027507543563842773 length of segment : 178 time for calcul the mask position with numpy : 0.001089334487915039 nb_pixel_total : 28016 time to create 1 rle with old method : 0.03162503242492676 length of segment : 194 time for calcul the mask position with numpy : 0.008903741836547852 nb_pixel_total : 274025 time to create 1 rle with new method : 0.014508962631225586 length of segment : 575 time for calcul the mask position with numpy : 0.0036497116088867188 nb_pixel_total : 29957 time to create 1 rle with old method : 0.03414344787597656 length of segment : 319 time for calcul the mask position with numpy : 0.0013837814331054688 nb_pixel_total : 32689 time to create 1 rle with old method : 0.0370938777923584 length of segment : 244 time for calcul the mask position with numpy : 0.0005061626434326172 nb_pixel_total : 16862 time to create 1 rle with old method : 0.0191190242767334 length of segment : 213 time for calcul the mask position with numpy : 0.009180068969726562 nb_pixel_total : 177577 time to create 1 rle with new method : 0.011985301971435547 length of segment : 1180 time for calcul the mask position with numpy : 0.012728452682495117 nb_pixel_total : 233988 time to create 1 rle with new method : 0.03403043746948242 length of segment : 909 time for calcul the mask position with numpy : 0.0010223388671875 nb_pixel_total : 41553 time to create 1 rle with old method : 0.04595136642456055 length of segment : 314 time for calcul the mask position with numpy : 0.0031065940856933594 nb_pixel_total : 117853 time to create 1 rle with old method : 0.13153409957885742 length of segment : 308 time for calcul the mask position with numpy : 0.0019795894622802734 nb_pixel_total : 46613 time to create 1 rle with old method : 0.05489635467529297 length of segment : 203 time for calcul the mask position with numpy : 0.0011005401611328125 nb_pixel_total : 29913 time to create 1 rle with old method : 0.03381705284118652 length of segment : 157 time for calcul the mask position with numpy : 0.002927541732788086 nb_pixel_total : 67641 time to create 1 rle with old method : 0.0742955207824707 length of segment : 526 time for calcul the mask position with numpy : 0.011136293411254883 nb_pixel_total : 239633 time to create 1 rle with new method : 0.030223608016967773 length of segment : 697 time for calcul the mask position with numpy : 0.003946781158447266 nb_pixel_total : 120401 time to create 1 rle with old method : 0.13553571701049805 length of segment : 525 time for calcul the mask position with numpy : 0.0013344287872314453 nb_pixel_total : 44477 time to create 1 rle with old method : 0.05004525184631348 length of segment : 250 time spent for convertir_results : 8.136302947998047 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 86 chid ids of type : 3594 Number RLEs to save : 27623 save missing photos in datou_result : time spend for datou_step_exec : 50.46975040435791 time spend to save output : 1.5772855281829834 total time spend for step 1 : 52.047035932540894 step2:crop_condition Thu May 29 13:11:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 86 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 50 About to insert : list_path_to_insert length 50 new photo from crops ! About to upload 50 photos upload in portfolio : 3736932 init cache_photo without model_param we have 50 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748517100_2294421 we have uploaded 50 photos in the portfolio 3736932 time of upload the photos Elapsed time : 12.613726377487183 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3736932 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748517115_2294421 we have uploaded 8 photos in the portfolio 3736932 time of upload the photos Elapsed time : 2.862778902053833 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748517119_2294421 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6056344509124756 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 27 About to insert : list_path_to_insert length 27 new photo from crops ! About to upload 27 photos upload in portfolio : 3736932 init cache_photo without model_param we have 27 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1748517129_2294421 we have uploaded 27 photos in the portfolio 3736932 time of upload the photos Elapsed time : 9.7368483543396 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1361620898, 1361618422, 1361618418, 1361618326, 1361618324, 1361618321, 1361618318, 1361618312] Looping around the photos to save general results len do output : 86 /1361624457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624547Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1361624553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361620898', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618422', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618418', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618326', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618324', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618321', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618318', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618312', None, None, None, None, None, '2983358') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 266 time used for this insertion : 0.03534674644470215 save_final save missing photos in datou_result : time spend for datou_step_exec : 58.19533920288086 time spend to save output : 0.037535905838012695 total time spend for step 2 : 58.23287510871887 step3:rle_unique_nms_with_priority Thu May 29 13:12:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 86 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ No data in photo_id : 1361620898 No data in photo_id : 1361618422 nb_obj : 12 nb_hashtags : 2 time to prepare the origin masks : 5.835540056228638 time for calcul the mask position with numpy : 0.4051978588104248 nb_pixel_total : 5909199 time to create 1 rle with new method : 0.6667816638946533 time for calcul the mask position with numpy : 0.04223155975341797 nb_pixel_total : 122152 time to create 1 rle with old method : 0.13339519500732422 time for calcul the mask position with numpy : 0.040709733963012695 nb_pixel_total : 94121 time to create 1 rle with old method : 0.10456418991088867 time for calcul the mask position with numpy : 0.04677462577819824 nb_pixel_total : 131681 time to create 1 rle with old method : 0.18596124649047852 time for calcul the mask position with numpy : 0.061608076095581055 nb_pixel_total : 24325 time to create 1 rle with old method : 0.02812504768371582 time for calcul the mask position with numpy : 0.0701298713684082 nb_pixel_total : 16160 time to create 1 rle with old method : 0.024811267852783203 time for calcul the mask position with numpy : 0.059110403060913086 nb_pixel_total : 45868 time to create 1 rle with old method : 0.05243277549743652 time for calcul the mask position with numpy : 0.05919361114501953 nb_pixel_total : 167876 time to create 1 rle with new method : 0.7480654716491699 time for calcul the mask position with numpy : 0.05627083778381348 nb_pixel_total : 289861 time to create 1 rle with new method : 0.5029094219207764 time for calcul the mask position with numpy : 0.055117130279541016 nb_pixel_total : 86514 time to create 1 rle with old method : 0.10959386825561523 time for calcul the mask position with numpy : 0.04329848289489746 nb_pixel_total : 41204 time to create 1 rle with old method : 0.04919314384460449 time for calcul the mask position with numpy : 0.05679440498352051 nb_pixel_total : 86877 time to create 1 rle with old method : 0.0965113639831543 time for calcul the mask position with numpy : 0.05705094337463379 nb_pixel_total : 34402 time to create 1 rle with old method : 0.03798246383666992 create new chi : 3.87518048286438 time to delete rle : 0.03064727783203125 batch 1 Loaded 25 chid ids of type : 3594 +++++++++++++++++++++Number RLEs to save : 11766 TO DO : save crop sub photo not yet done ! save time : 0.719566822052002 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 4.015625238418579 time for calcul the mask position with numpy : 0.5557825565338135 nb_pixel_total : 6750037 time to create 1 rle with new method : 0.5101227760314941 time for calcul the mask position with numpy : 0.03903055191040039 nb_pixel_total : 10915 time to create 1 rle with old method : 0.011910676956176758 time for calcul the mask position with numpy : 0.03889322280883789 nb_pixel_total : 59838 time to create 1 rle with old method : 0.06598734855651855 time for calcul the mask position with numpy : 0.05184364318847656 nb_pixel_total : 76136 time to create 1 rle with old method : 0.08251428604125977 time for calcul the mask position with numpy : 0.0520021915435791 nb_pixel_total : 9873 time to create 1 rle with old method : 0.010856151580810547 time for calcul the mask position with numpy : 0.05661416053771973 nb_pixel_total : 29892 time to create 1 rle with old method : 0.031829833984375 time for calcul the mask position with numpy : 0.05328536033630371 nb_pixel_total : 15945 time to create 1 rle with old method : 0.017586231231689453 time for calcul the mask position with numpy : 0.03921151161193848 nb_pixel_total : 35658 time to create 1 rle with old method : 0.03753089904785156 time for calcul the mask position with numpy : 0.03756546974182129 nb_pixel_total : 18319 time to create 1 rle with old method : 0.019129276275634766 time for calcul the mask position with numpy : 0.03711342811584473 nb_pixel_total : 43627 time to create 1 rle with old method : 0.04567861557006836 create new chi : 1.8331210613250732 time to delete rle : 0.0009813308715820312 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++Number RLEs to save : 6288 TO DO : save crop sub photo not yet done ! save time : 0.39692211151123047 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 4.034874439239502 time for calcul the mask position with numpy : 0.6138837337493896 nb_pixel_total : 6505189 time to create 1 rle with new method : 1.0084128379821777 time for calcul the mask position with numpy : 0.03953433036804199 nb_pixel_total : 30490 time to create 1 rle with old method : 0.03339123725891113 time for calcul the mask position with numpy : 0.040223121643066406 nb_pixel_total : 76949 time to create 1 rle with old method : 0.08475518226623535 time for calcul the mask position with numpy : 0.04027438163757324 nb_pixel_total : 61153 time to create 1 rle with old method : 0.06703805923461914 time for calcul the mask position with numpy : 0.04198932647705078 nb_pixel_total : 103497 time to create 1 rle with old method : 0.11265206336975098 time for calcul the mask position with numpy : 0.04063677787780762 nb_pixel_total : 76689 time to create 1 rle with old method : 0.08373260498046875 time for calcul the mask position with numpy : 0.04071521759033203 nb_pixel_total : 27027 time to create 1 rle with old method : 0.0300900936126709 time for calcul the mask position with numpy : 0.04069328308105469 nb_pixel_total : 47912 time to create 1 rle with old method : 0.053478240966796875 time for calcul the mask position with numpy : 0.041681766510009766 nb_pixel_total : 26935 time to create 1 rle with old method : 0.03169655799865723 time for calcul the mask position with numpy : 0.04188251495361328 nb_pixel_total : 15828 time to create 1 rle with old method : 0.017676591873168945 time for calcul the mask position with numpy : 0.04508638381958008 nb_pixel_total : 32913 time to create 1 rle with old method : 0.03674626350402832 time for calcul the mask position with numpy : 0.05370521545410156 nb_pixel_total : 45658 time to create 1 rle with old method : 0.05009150505065918 create new chi : 2.7287604808807373 time to delete rle : 0.0012924671173095703 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++Number RLEs to save : 7974 TO DO : save crop sub photo not yet done ! save time : 0.4890787601470947 nb_obj : 22 nb_hashtags : 3 time to prepare the origin masks : 6.537256240844727 time for calcul the mask position with numpy : 0.2789642810821533 nb_pixel_total : 6089808 time to create 1 rle with new method : 0.4085409641265869 time for calcul the mask position with numpy : 0.05497002601623535 nb_pixel_total : 197790 time to create 1 rle with new method : 0.47144412994384766 time for calcul the mask position with numpy : 0.0516359806060791 nb_pixel_total : 10668 time to create 1 rle with old method : 0.011327743530273438 time for calcul the mask position with numpy : 0.04446053504943848 nb_pixel_total : 19965 time to create 1 rle with old method : 0.021382808685302734 time for calcul the mask position with numpy : 0.0399775505065918 nb_pixel_total : 8185 time to create 1 rle with old method : 0.009130239486694336 time for calcul the mask position with numpy : 0.03903985023498535 nb_pixel_total : 16472 time to create 1 rle with old method : 0.023348331451416016 time for calcul the mask position with numpy : 0.042569637298583984 nb_pixel_total : 50194 time to create 1 rle with old method : 0.06332969665527344 time for calcul the mask position with numpy : 0.04002737998962402 nb_pixel_total : 12399 time to create 1 rle with old method : 0.013214111328125 time for calcul the mask position with numpy : 0.038095951080322266 nb_pixel_total : 25943 time to create 1 rle with old method : 0.026803970336914062 time for calcul the mask position with numpy : 0.03894829750061035 nb_pixel_total : 25271 time to create 1 rle with old method : 0.027121543884277344 time for calcul the mask position with numpy : 0.04045224189758301 nb_pixel_total : 16036 time to create 1 rle with old method : 0.01816248893737793 time for calcul the mask position with numpy : 0.04789590835571289 nb_pixel_total : 26343 time to create 1 rle with old method : 0.03343629837036133 time for calcul the mask position with numpy : 0.040232181549072266 nb_pixel_total : 49455 time to create 1 rle with old method : 0.05289435386657715 time for calcul the mask position with numpy : 0.03966832160949707 nb_pixel_total : 164003 time to create 1 rle with new method : 0.43128299713134766 time for calcul the mask position with numpy : 0.040790557861328125 nb_pixel_total : 50559 time to create 1 rle with old method : 0.053925275802612305 time for calcul the mask position with numpy : 0.04278993606567383 nb_pixel_total : 4563 time to create 1 rle with old method : 0.005112409591674805 time for calcul the mask position with numpy : 0.040575265884399414 nb_pixel_total : 14804 time to create 1 rle with old method : 0.016679048538208008 time for calcul the mask position with numpy : 0.03977560997009277 nb_pixel_total : 26335 time to create 1 rle with old method : 0.02849888801574707 time for calcul the mask position with numpy : 0.03978276252746582 nb_pixel_total : 43380 time to create 1 rle with old method : 0.04743552207946777 time for calcul the mask position with numpy : 0.03829598426818848 nb_pixel_total : 36268 time to create 1 rle with old method : 0.03917837142944336 time for calcul the mask position with numpy : 0.03930950164794922 nb_pixel_total : 14602 time to create 1 rle with old method : 0.01582646369934082 time for calcul the mask position with numpy : 0.04053068161010742 nb_pixel_total : 126193 time to create 1 rle with old method : 0.16330480575561523 time for calcul the mask position with numpy : 0.04065299034118652 nb_pixel_total : 21004 time to create 1 rle with old method : 0.023012876510620117 create new chi : 3.2830452919006348 time to delete rle : 0.0025110244750976562 batch 1 Loaded 45 chid ids of type : 3594 ++++++++++++++++++++++++++++Number RLEs to save : 12964 TO DO : save crop sub photo not yet done ! save time : 0.7055723667144775 nb_obj : 15 nb_hashtags : 4 time to prepare the origin masks : 7.497858047485352 time for calcul the mask position with numpy : 0.861135721206665 nb_pixel_total : 5866479 time to create 1 rle with new method : 0.9201250076293945 time for calcul the mask position with numpy : 0.050241708755493164 nb_pixel_total : 15584 time to create 1 rle with old method : 0.016610383987426758 time for calcul the mask position with numpy : 0.052722930908203125 nb_pixel_total : 41135 time to create 1 rle with old method : 0.04448246955871582 time for calcul the mask position with numpy : 0.05543017387390137 nb_pixel_total : 316704 time to create 1 rle with new method : 0.6186678409576416 time for calcul the mask position with numpy : 0.05338263511657715 nb_pixel_total : 35029 time to create 1 rle with old method : 0.03759288787841797 time for calcul the mask position with numpy : 0.051847219467163086 nb_pixel_total : 62391 time to create 1 rle with old method : 0.07285571098327637 time for calcul the mask position with numpy : 0.05747246742248535 nb_pixel_total : 43892 time to create 1 rle with old method : 0.05182480812072754 time for calcul the mask position with numpy : 0.05287504196166992 nb_pixel_total : 139234 time to create 1 rle with old method : 0.1788949966430664 time for calcul the mask position with numpy : 0.054345130920410156 nb_pixel_total : 92662 time to create 1 rle with old method : 0.09879660606384277 time for calcul the mask position with numpy : 0.04861569404602051 nb_pixel_total : 29208 time to create 1 rle with old method : 0.031122446060180664 time for calcul the mask position with numpy : 0.04036116600036621 nb_pixel_total : 154784 time to create 1 rle with new method : 0.5811426639556885 time for calcul the mask position with numpy : 0.05225038528442383 nb_pixel_total : 68149 time to create 1 rle with old method : 0.0736541748046875 time for calcul the mask position with numpy : 0.05243968963623047 nb_pixel_total : 26265 time to create 1 rle with old method : 0.028857707977294922 time for calcul the mask position with numpy : 0.05416536331176758 nb_pixel_total : 14861 time to create 1 rle with old method : 0.01646709442138672 time for calcul the mask position with numpy : 0.054787397384643555 nb_pixel_total : 104233 time to create 1 rle with old method : 0.11456513404846191 time for calcul the mask position with numpy : 0.05449867248535156 nb_pixel_total : 39630 time to create 1 rle with old method : 0.04118704795837402 create new chi : 4.6580421924591064 time to delete rle : 0.002512693405151367 batch 1 Loaded 31 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 12773 TO DO : save crop sub photo not yet done ! save time : 0.76363205909729 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 5.593563795089722 time for calcul the mask position with numpy : 0.4855000972747803 nb_pixel_total : 5659088 time to create 1 rle with new method : 0.49659085273742676 time for calcul the mask position with numpy : 0.050205230712890625 nb_pixel_total : 44477 time to create 1 rle with old method : 0.04678630828857422 time for calcul the mask position with numpy : 0.05147075653076172 nb_pixel_total : 120401 time to create 1 rle with old method : 0.12443280220031738 time for calcul the mask position with numpy : 0.049697160720825195 nb_pixel_total : 45905 time to create 1 rle with old method : 0.047394514083862305 time for calcul the mask position with numpy : 0.050827980041503906 nb_pixel_total : 67641 time to create 1 rle with old method : 0.07137680053710938 time for calcul the mask position with numpy : 0.050691843032836914 nb_pixel_total : 29913 time to create 1 rle with old method : 0.031148433685302734 time for calcul the mask position with numpy : 0.04977560043334961 nb_pixel_total : 46613 time to create 1 rle with old method : 0.0491487979888916 time for calcul the mask position with numpy : 0.03918337821960449 nb_pixel_total : 117853 time to create 1 rle with old method : 0.12249255180358887 time for calcul the mask position with numpy : 0.03930258750915527 nb_pixel_total : 39749 time to create 1 rle with old method : 0.04192686080932617 time for calcul the mask position with numpy : 0.04117536544799805 nb_pixel_total : 233988 time to create 1 rle with new method : 0.5801410675048828 time for calcul the mask position with numpy : 0.03828072547912598 nb_pixel_total : 177577 time to create 1 rle with new method : 0.6444623470306396 time for calcul the mask position with numpy : 0.03774666786193848 nb_pixel_total : 16862 time to create 1 rle with old method : 0.017173051834106445 time for calcul the mask position with numpy : 0.0372316837310791 nb_pixel_total : 32579 time to create 1 rle with old method : 0.0335848331451416 time for calcul the mask position with numpy : 0.04068565368652344 nb_pixel_total : 29957 time to create 1 rle with old method : 0.030883073806762695 time for calcul the mask position with numpy : 0.041433095932006836 nb_pixel_total : 274025 time to create 1 rle with new method : 0.4644496440887451 time for calcul the mask position with numpy : 0.04080033302307129 nb_pixel_total : 28016 time to create 1 rle with old method : 0.03011798858642578 time for calcul the mask position with numpy : 0.0511322021484375 nb_pixel_total : 24346 time to create 1 rle with old method : 0.02477288246154785 time for calcul the mask position with numpy : 0.04978466033935547 nb_pixel_total : 61250 time to create 1 rle with old method : 0.06349468231201172 create new chi : 4.265686511993408 time to delete rle : 0.0019617080688476562 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++++++Number RLEs to save : 15921 TO DO : save crop sub photo not yet done ! save time : 0.9291484355926514 map_output_result : {1361620898: (0.0, 'Should be the crop_list due to order', 0.0), 1361618422: (0.0, 'Should be the crop_list due to order', 0.0), 1361618418: (0.0, 'Should be the crop_list due to order', 0), 1361618326: (0.0, 'Should be the crop_list due to order', 0), 1361618324: (0.0, 'Should be the crop_list due to order', 0), 1361618321: (0.0, 'Should be the crop_list due to order', 0), 1361618318: (0.0, 'Should be the crop_list due to order', 0), 1361618312: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1361620898, 1361618422, 1361618418, 1361618326, 1361618324, 1361618321, 1361618318, 1361618312] Looping around the photos to save general results len do output : 8 /1361620898.Didn't retrieve data . /1361618422.Didn't retrieve data . /1361618418.Didn't retrieve data . /1361618326.Didn't retrieve data . /1361618324.Didn't retrieve data . /1361618321.Didn't retrieve data . /1361618318.Didn't retrieve data . /1361618312.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361620898', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618422', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618418', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618326', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618324', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618321', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618318', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618312', None, None, None, None, None, '2983358') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.011893510818481445 save_final save missing photos in datou_result : time spend for datou_step_exec : 59.40498638153076 time spend to save output : 0.012278556823730469 total time spend for step 3 : 59.41726493835449 step4:ventilate_hashtags_in_portfolio Thu May 29 13:13:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23439680 get user id for portfolio 23439680 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23439680 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','papier','pet_clair','carton','environnement','mal_croppe','autre','pet_fonce','metal','pehd','flou')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23439680 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','papier','pet_clair','carton','environnement','mal_croppe','autre','pet_fonce','metal','pehd','flou')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23439680 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('background','papier','pet_clair','carton','environnement','mal_croppe','autre','pet_fonce','metal','pehd','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23439776,23439777,23439778,23439779,23439780,23439781,23439782,23439783,23439784,23439785,23439786?tags=background,papier,pet_clair,carton,environnement,mal_croppe,autre,pet_fonce,metal,pehd,flou Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1361620898, 1361618422, 1361618418, 1361618326, 1361618324, 1361618321, 1361618318, 1361618312] Looping around the photos to save general results len do output : 1 /23439680. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361620898', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618422', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618418', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618326', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618324', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618321', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618318', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618312', None, None, None, None, None, '2983358') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013721466064453125 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.5532500743865967 time spend to save output : 0.013965368270874023 total time spend for step 4 : 1.5672154426574707 step5:final Thu May 29 13:13:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1361620898: ('0.09789808573892522',), 1361618422: ('0.09789808573892522',), 1361618418: ('0.09789808573892522',), 1361618326: ('0.09789808573892522',), 1361618324: ('0.09789808573892522',), 1361618321: ('0.09789808573892522',), 1361618318: ('0.09789808573892522',), 1361618312: ('0.09789808573892522',)} new output for save of step final : {1361620898: ('0.09789808573892522',), 1361618422: ('0.09789808573892522',), 1361618418: ('0.09789808573892522',), 1361618326: ('0.09789808573892522',), 1361618324: ('0.09789808573892522',), 1361618321: ('0.09789808573892522',), 1361618318: ('0.09789808573892522',), 1361618312: ('0.09789808573892522',)} [1361620898, 1361618422, 1361618418, 1361618326, 1361618324, 1361618321, 1361618318, 1361618312] Looping around the photos to save general results len do output : 8 /1361620898.Didn't retrieve data . /1361618422.Didn't retrieve data . /1361618418.Didn't retrieve data . /1361618326.Didn't retrieve data . /1361618324.Didn't retrieve data . /1361618321.Didn't retrieve data . /1361618318.Didn't retrieve data . /1361618312.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361620898', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618422', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618418', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618326', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618324', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618321', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618318', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618312', None, None, None, None, None, '2983358') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.012444734573364258 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1058511734008789 time spend to save output : 0.012896299362182617 total time spend for step 5 : 0.11874747276306152 step6:blur_detection Thu May 29 13:13:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1748517027_2294421_1361620898_2982070e5c0bc8af62e710c2bd9ceba3.jpg resize: (2160, 3264) 1361620898 1.2986178555473784 treat image : temp/1748517027_2294421_1361618422_c3b41996025eb29beeb26e6536aed7f7.jpg resize: (2160, 3264) 1361618422 0.36323881466580066 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be.jpg resize: (2160, 3264) 1361618418 2.789180346058731 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4.jpg resize: (2160, 3264) 1361618326 1.3738305114420475 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464.jpg resize: (2160, 3264) 1361618324 -2.2431583238124166 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf.jpg resize: (2160, 3264) 1361618321 -2.4227083536309464 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8.jpg resize: (2160, 3264) 1361618318 1.446999963974181 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de.jpg resize: (2160, 3264) 1361618312 -1.2132383603076042 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192347_0.png resize: (247, 225) 1361624457 -1.8203204281355332 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192350_0.png resize: (308, 420) 1361624458 -0.5523005242359571 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192352_0.png resize: (495, 493) 1361624459 1.0320711566722471 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192354_0.png resize: (157, 204) 1361624460 -0.39181728350327505 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192355_0.png resize: (301, 137) 1361624461 -2.3086233981479483 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192356_0.png resize: (426, 478) 1361624462 0.9006349839405033 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192357_0.png resize: (363, 350) 1361624463 0.8553554240289477 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192358_0.png resize: (449, 610) 1361624464 -0.04554890942012923 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192359_0.png resize: (249, 252) 1361624465 0.29123818412420294 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192360_0.png resize: (220, 207) 1361624466 0.3469363539192952 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192361_0.png resize: (233, 226) 1361624467 -0.46559557378074595 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192362_0.png resize: (141, 205) 1361624468 -0.9943038374534547 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192363_0.png resize: (197, 186) 1361624469 0.47541177779007365 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192364_0.png resize: (119, 110) 1361624470 2.2140652801623375 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192365_0.png resize: (363, 322) 1361624471 -1.3546122461730494 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192367_0.png resize: (120, 133) 1361624472 0.008252452957064041 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192368_0.png resize: (229, 347) 1361624473 -0.9042684348767299 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192369_0.png resize: (150, 303) 1361624474 -0.5695715352120345 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192371_0.png resize: (247, 244) 1361624475 -1.3966720757152256 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192372_0.png resize: (293, 286) 1361624476 -1.4875810466746096 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192373_0.png resize: (199, 182) 1361624477 -1.5510958567818431 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192374_0.png resize: (448, 281) 1361624478 -1.2369509862387154 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192375_0.png resize: (318, 455) 1361624479 -1.714020885878212 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192376_0.png resize: (287, 347) 1361624481 -3.0041141485729135 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192379_0.png resize: (132, 217) 1361624482 -0.733362196076066 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192382_0.png resize: (184, 261) 1361624483 -1.3577011061702275 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192384_0.png resize: (241, 166) 1361624484 -0.8688271868317803 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192385_0.png resize: (114, 153) 1361624485 0.002509347459531971 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192386_0.png resize: (125, 45) 1361624486 1.5212215560548712 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192388_0.png resize: (477, 551) 1361624487 -2.0446111888990606 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192389_0.png resize: (304, 232) 1361624488 -2.006778062741417 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192390_0.png resize: (183, 228) 1361624489 -1.228982090723248 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192391_0.png resize: (200, 201) 1361624490 -1.7632653878334812 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192397_0.png resize: (64, 171) 1361624491 -0.8938620972972092 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192398_0.png resize: (201, 194) 1361624492 -1.15358466814644 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192400_0.png resize: (650, 436) 1361624493 -1.934458631891253 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192401_0.png resize: (297, 177) 1361624494 0.6690278070116094 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192402_0.png resize: (412, 425) 1361624495 -0.8044172042013512 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192408_0.png resize: (396, 338) 1361624496 0.5029537361956352 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192409_0.png resize: (886, 296) 1361624497 -0.48128747399505284 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192413_0.png resize: (627, 753) 1361624498 -0.04170578628439731 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192415_0.png resize: (285, 694) 1361624499 0.5702125494065157 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192417_0.png resize: (177, 175) 1361624500 -0.16995037821260905 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192418_0.png resize: (194, 199) 1361624501 -0.5869296978375499 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192420_0.png resize: (333, 499) 1361624502 -0.9986921503019015 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192422_0.png resize: (213, 115) 1361624503 -1.5981642924071784 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192423_0.png resize: (1165, 225) 1361624504 -1.1034362568339082 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192426_0.png resize: (293, 586) 1361624505 -1.6266158473528205 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192430_0.png resize: (662, 734) 1361624506 -0.6082089975221152 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192431_0.png resize: (441, 344) 1361624507 -0.8695600352141907 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192370_0.png resize: (169, 123) 1361624509 -0.439708648029956 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192380_0.png resize: (476, 313) 1361624510 -0.5799680330953697 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192381_0.png resize: (143, 144) 1361624511 -1.2382177561188246 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192393_0.png resize: (190, 196) 1361624512 -1.5819151472581685 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192396_0.png resize: (135, 177) 1361624513 -1.5298278724705745 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192404_0.png resize: (207, 219) 1361624514 -0.6522208383186358 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192414_0.png resize: (232, 255) 1361624515 -0.5414694026047981 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192424_0.png resize: (808, 667) 1361624516 -0.8394940526504914 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192403_0.png resize: (132, 131) 1361624518 -0.7602271688713216 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192348_0.png resize: (464, 340) 1361624527 -0.5102620827854398 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192349_0.png resize: (161, 362) 1361624528 -1.7013307032840645 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192351_0.png resize: (461, 925) 1361624529 -0.7480574744712606 treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192353_0.png resize: (403, 329) 1361624530 -2.764633896614527 treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192366_0.png resize: (340, 203) 1361624531 0.351221551667712 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192377_0.png resize: (285, 442) 1361624532 -2.664987130726228 treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192378_0.png resize: (194, 208) 1361624533 -2.1390044830273407 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192383_0.png resize: (225, 257) 1361624534 -2.1810335803457486 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192387_0.png resize: (216, 295) 1361624535 -1.1615212979579497 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192392_0.png resize: (247, 171) 1361624536 -2.212853869180099 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192394_0.png resize: (115, 131) 1361624537 -2.487740697200709 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192395_0.png resize: (258, 274) 1361624538 -1.8336601588875534 treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192399_0.png resize: (93, 152) 1361624539 -2.4340267683503156 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192405_0.png resize: (213, 394) 1361624540 -0.332262390071148 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192406_0.png resize: (273, 609) 1361624541 1.9627892986417972 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192407_0.png resize: (226, 239) 1361624542 -0.4111338057739277 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192410_0.png resize: (185, 375) 1361624543 -1.782676658410908 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192411_0.png resize: (421, 229) 1361624544 -1.3455609387362246 treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192412_0.png resize: (209, 236) 1361624545 -0.52076001949023 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192416_0.png resize: (231, 334) 1361624546 -0.8756387026085689 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192419_0.png resize: (522, 706) 1361624547 -2.9527640098917374 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192421_0.png resize: (223, 269) 1361624548 -1.8955976702361252 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192425_0.png resize: (312, 181) 1361624549 -2.4359431127427134 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192427_0.png resize: (191, 400) 1361624550 -1.6243614713007528 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192428_0.png resize: (155, 221) 1361624551 0.4108307149753105 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192429_0.png resize: (332, 311) 1361624552 -1.798036055519156 treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192432_0.png resize: (159, 383) 1361624553 -1.756227114071504 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 94 time used for this insertion : 0.019699811935424805 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 94 time used for this insertion : 0.02147817611694336 save missing photos in datou_result : time spend for datou_step_exec : 31.629759550094604 time spend to save output : 0.0462491512298584 total time spend for step 6 : 31.676008701324463 step7:brightness Thu May 29 13:13:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1748517027_2294421_1361620898_2982070e5c0bc8af62e710c2bd9ceba3.jpg treat image : temp/1748517027_2294421_1361618422_c3b41996025eb29beeb26e6536aed7f7.jpg treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be.jpg treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4.jpg treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464.jpg treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf.jpg treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8.jpg treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de.jpg treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192347_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192350_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192352_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192354_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192355_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192356_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192357_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192358_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192359_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192360_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192361_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192362_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192363_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192364_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192365_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192367_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192368_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192369_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192371_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192372_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192373_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192374_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192375_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192376_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192379_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192382_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192384_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192385_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192386_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192388_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192389_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192390_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192391_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192397_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192398_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192400_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192401_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192402_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192408_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192409_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192413_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192415_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192417_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192418_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192420_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192422_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192423_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192426_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192430_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192431_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192370_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192380_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192381_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192393_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192396_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192404_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192414_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192424_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192403_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192348_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192349_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192351_0.png treat image : temp/1748517027_2294421_1361618418_d263bf52851afa3aa396950abb0462be_rle_crop_3815192353_0.png treat image : temp/1748517027_2294421_1361618326_f0102de9777703f3f688e91afd7b07e4_rle_crop_3815192366_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192377_0.png treat image : temp/1748517027_2294421_1361618324_85d672d531853a76dccde420d18c4464_rle_crop_3815192378_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192383_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192387_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192392_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192394_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192395_0.png treat image : temp/1748517027_2294421_1361618321_e61485fe0c8a0ebaf9dc7ef215110adf_rle_crop_3815192399_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192405_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192406_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192407_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192410_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192411_0.png treat image : temp/1748517027_2294421_1361618318_9ea6c1a3582ef32097f3e8d4b85fd8d8_rle_crop_3815192412_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192416_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192419_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192421_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192425_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192427_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192428_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192429_0.png treat image : temp/1748517027_2294421_1361618312_b3d8c7337693d690963eaffce58f95de_rle_crop_3815192432_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 94 time used for this insertion : 0.020145177841186523 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 94 time used for this insertion : 0.01733994483947754 save missing photos in datou_result : time spend for datou_step_exec : 7.819584608078003 time spend to save output : 0.043056488037109375 total time spend for step 7 : 7.862641096115112 step8:velours_tree Thu May 29 13:13:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.13833308219909668 time spend to save output : 6.008148193359375e-05 total time spend for step 8 : 0.13839316368103027 step9:send_mail_cod Thu May 29 13:14:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23439680_29-05-2025_13_14_00.pdf 23439776 imagette234397761748517240 23439777 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234397771748517240 23439778 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234397781748517241 23439779 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette234397791748517242 23439781 imagette234397811748517243 23439782 imagette234397821748517243 23439783 imagette234397831748517243 23439784 change filename to text .imagette234397841748517243 23439785 imagette234397851748517243 23439786 imagette234397861748517243 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23439680 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23439776,23439777,23439778,23439779,23439780,23439781,23439782,23439783,23439784,23439785,23439786?tags=background,papier,pet_clair,carton,environnement,mal_croppe,autre,pet_fonce,metal,pehd,flou args[1361620898] : ((1361620898, 1.2986178555473784, 492688767), (1361620898, 0.1493626642790122, 2107752395), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618422] : ((1361618422, 0.36323881466580066, 492688767), (1361618422, -0.20881291648522424, 496442774), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618418] : ((1361618418, 2.789180346058731, 492688767), (1361618418, -0.1535624103147365, 496442774), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618326] : ((1361618326, 1.3738305114420475, 492688767), (1361618326, 0.2937096521195026, 2107752395), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618324] : ((1361618324, -2.2431583238124166, 492609224), (1361618324, 0.3926623496256716, 2107752395), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618321] : ((1361618321, -2.4227083536309464, 492609224), (1361618321, 0.05914928352171113, 2107752395), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618318] : ((1361618318, 1.446999963974181, 492688767), (1361618318, 0.24857333527549239, 2107752395), '0.09789808573892522') We are sending mail with results at report@fotonower.com args[1361618312] : ((1361618312, -1.2132383603076042, 492688767), (1361618312, 0.08838595201931833, 2107752395), '0.09789808573892522') We are sending mail with results at report@fotonower.com refus_total : 0.09789808573892522 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23439680 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439680_29-05-2025_13_14_00.pdf results_Auto_P23439680_29-05-2025_13_14_00.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439680_29-05-2025_13_14_00.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23439680','results_Auto_P23439680_29-05-2025_13_14_00.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439680_29-05-2025_13_14_00.pdf','pdf','','0.6','0.09789808573892522') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23439680

https://www.fotonower.com/image?json=false&list_photos_id=1361620898
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.2986178555473784)
https://www.fotonower.com/image?json=false&list_photos_id=1361618422
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361618418
La photo est trop floue, merci de reprendre une photo.(avec le score = 2.789180346058731)
https://www.fotonower.com/image?json=false&list_photos_id=1361618326
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.3738305114420475)
https://www.fotonower.com/image?json=false&list_photos_id=1361618324
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361618321
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1361618318
La photo est trop floue, merci de reprendre une photo.(avec le score = 1.446999963974181)
https://www.fotonower.com/image?json=false&list_photos_id=1361618312
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 9.79%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/23439777?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23439778?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23439779?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/23439784?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439680_29-05-2025_13_14_00.pdf.

Lien vers velours :https://www.fotonower.com/velours/23439776,23439777,23439778,23439779,23439780,23439781,23439782,23439783,23439784,23439785,23439786?tags=background,papier,pet_clair,carton,environnement,mal_croppe,autre,pet_fonce,metal,pehd,flou.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 29 May 2025 11:14:05 GMT Content-Length: 0 Connection: close X-Message-Id: DzUeCuzUQ_SI8gm2At3CXQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1361620898, 1361618422, 1361618418, 1361618326, 1361618324, 1361618321, 1361618318, 1361618312] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361620898', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618422', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618418', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618326', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618324', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618321', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618318', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618312', None, None, None, None, None, '2983358') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.015630006790161133 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.794588088989258 time spend to save output : 0.015814542770385742 total time spend for step 9 : 5.8104026317596436 step10:split_time_score Thu May 29 13:14:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 29052025 23439680 Nombre de photos uploadées : 8 / 23040 (0%) 29052025 23439680 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 29052025 23439680 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 3.5762786865234375e-06 elapsed_time : order_list_meta_photo_and_scores 1.049041748046875e-05 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0005323886871337891 elapsed_time : insert_dashboard_record_day_entry 0.02219986915588379 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23438656 order by id desc limit 1 Qualite : 0.04350078289124783 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23438660_29-05-2025_10_20_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23438660 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23438660 AND mptpi.`type`=3726 To do Qualite : 0.13861518898647426 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439179_29-05-2025_11_31_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23439179 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23439179 AND mptpi.`type`=3594 To do Qualite : 0.09789808573892522 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23439680_29-05-2025_13_14_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23439680 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23439680 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'29052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1361620898, 1361618422, 1361618418, 1361618326, 1361618324, 1361618321, 1361618318, 1361618312] Looping around the photos to save general results len do output : 1 /23439680Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361620898', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618422', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618418', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618326', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618324', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618321', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618318', None, None, None, None, None, '2983358') ('3318', None, None, None, None, None, None, None, '2983358') ('3318', '23439680', '1361618312', None, None, None, None, None, '2983358') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013477087020874023 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.2313156127929688 time spend to save output : 0.013657093048095703 total time spend for step 10 : 1.2449727058410645 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 101.88user 85.78system 3:41.75elapsed 84%CPU (0avgtext+0avgdata 4381680maxresident)k 538712inputs+62896outputs (184major+8649950minor)pagefaults 0swaps