python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2512277 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2935231'] with mtr_portfolio_ids : ['23263226'] and first list_photo_ids : [] new path : /proc/2512277/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 8 ; length of list_pids : 8 ; length of list_args : 8 time to download the photos : 1.263016939163208 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Thu May 22 11:10:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-22 11:10:32.018008: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-22 11:10:32.043254: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-22 11:10:32.044885: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8724000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-22 11:10:32.044937: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-22 11:10:32.047534: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-22 11:10:32.272230: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x8e175e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-22 11:10:32.272281: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-22 11:10:32.273775: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-22 11:10:32.274220: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-22 11:10:32.277681: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-22 11:10:32.280737: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-22 11:10:32.281158: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-22 11:10:32.284068: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-22 11:10:32.285501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-22 11:10:32.290602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-22 11:10:32.292137: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-22 11:10:32.292219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-22 11:10:32.293003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-22 11:10:32.293019: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-22 11:10:32.293029: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-22 11:10:32.294321: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-22 11:10:32.545931: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-22 11:10:32.546085: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-22 11:10:32.546120: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-22 11:10:32.546151: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-22 11:10:32.546182: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-22 11:10:32.546211: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-22 11:10:32.546241: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-22 11:10:32.546271: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-22 11:10:32.548350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-22 11:10:32.549542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-22 11:10:32.549579: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-22 11:10:32.549595: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-22 11:10:32.549608: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-22 11:10:32.549622: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-22 11:10:32.549636: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-22 11:10:32.549650: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-22 11:10:32.549664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-22 11:10:32.550865: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-22 11:10:32.550910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-22 11:10:32.550920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-22 11:10:32.550928: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-22 11:10:32.552142: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-22 11:10:42.745137: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-22 11:10:42.928079: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 45.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 41 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 30.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 37 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 16.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 14.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 13.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 Detection mask done ! Trying to reset tf kernel 2512876 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 8 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 8 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0005803108215332031 nb_pixel_total : 13501 time to create 1 rle with old method : 0.02042365074157715 length of segment : 205 time for calcul the mask position with numpy : 0.003913402557373047 nb_pixel_total : 119446 time to create 1 rle with old method : 0.18239307403564453 length of segment : 583 time for calcul the mask position with numpy : 0.0004086494445800781 nb_pixel_total : 23231 time to create 1 rle with old method : 0.027235031127929688 length of segment : 174 time for calcul the mask position with numpy : 0.0003871917724609375 nb_pixel_total : 26369 time to create 1 rle with old method : 0.03099226951599121 length of segment : 192 time for calcul the mask position with numpy : 0.00038170814514160156 nb_pixel_total : 22291 time to create 1 rle with old method : 0.026607990264892578 length of segment : 226 time for calcul the mask position with numpy : 0.006149768829345703 nb_pixel_total : 261701 time to create 1 rle with new method : 0.013868331909179688 length of segment : 677 time for calcul the mask position with numpy : 0.0001850128173828125 nb_pixel_total : 10373 time to create 1 rle with old method : 0.012668609619140625 length of segment : 71 time for calcul the mask position with numpy : 0.00014972686767578125 nb_pixel_total : 9502 time to create 1 rle with old method : 0.011880159378051758 length of segment : 87 time for calcul the mask position with numpy : 0.0003139972686767578 nb_pixel_total : 10752 time to create 1 rle with old method : 0.012836456298828125 length of segment : 202 time for calcul the mask position with numpy : 0.00038504600524902344 nb_pixel_total : 21407 time to create 1 rle with old method : 0.03026127815246582 length of segment : 237 time for calcul the mask position with numpy : 0.0006191730499267578 nb_pixel_total : 28933 time to create 1 rle with old method : 0.03306174278259277 length of segment : 170 time for calcul the mask position with numpy : 0.0057375431060791016 nb_pixel_total : 281333 time to create 1 rle with new method : 0.022452116012573242 length of segment : 693 time for calcul the mask position with numpy : 0.0003924369812011719 nb_pixel_total : 17119 time to create 1 rle with old method : 0.019510984420776367 length of segment : 213 time for calcul the mask position with numpy : 0.0012869834899902344 nb_pixel_total : 55142 time to create 1 rle with old method : 0.06011056900024414 length of segment : 494 time for calcul the mask position with numpy : 0.00041937828063964844 nb_pixel_total : 12063 time to create 1 rle with old method : 0.014060020446777344 length of segment : 145 time for calcul the mask position with numpy : 0.0010044574737548828 nb_pixel_total : 37167 time to create 1 rle with old method : 0.043048858642578125 length of segment : 346 time for calcul the mask position with numpy : 0.0004987716674804688 nb_pixel_total : 19641 time to create 1 rle with old method : 0.022162437438964844 length of segment : 122 time for calcul the mask position with numpy : 0.0026798248291015625 nb_pixel_total : 101549 time to create 1 rle with old method : 0.1127176284790039 length of segment : 586 time for calcul the mask position with numpy : 0.00033926963806152344 nb_pixel_total : 11627 time to create 1 rle with old method : 0.01343536376953125 length of segment : 145 time for calcul the mask position with numpy : 0.0005147457122802734 nb_pixel_total : 16340 time to create 1 rle with old method : 0.018661975860595703 length of segment : 228 time for calcul the mask position with numpy : 0.0015904903411865234 nb_pixel_total : 67389 time to create 1 rle with old method : 0.07534432411193848 length of segment : 292 time for calcul the mask position with numpy : 0.0012614727020263672 nb_pixel_total : 49643 time to create 1 rle with old method : 0.056177616119384766 length of segment : 308 time for calcul the mask position with numpy : 0.0011281967163085938 nb_pixel_total : 45105 time to create 1 rle with old method : 0.050261497497558594 length of segment : 304 time for calcul the mask position with numpy : 0.0006906986236572266 nb_pixel_total : 31072 time to create 1 rle with old method : 0.03450775146484375 length of segment : 228 time for calcul the mask position with numpy : 0.003195524215698242 nb_pixel_total : 146479 time to create 1 rle with old method : 0.15958404541015625 length of segment : 696 time for calcul the mask position with numpy : 0.0005009174346923828 nb_pixel_total : 11195 time to create 1 rle with old method : 0.01331186294555664 length of segment : 138 time for calcul the mask position with numpy : 0.0004417896270751953 nb_pixel_total : 16989 time to create 1 rle with old method : 0.019748926162719727 length of segment : 146 time for calcul the mask position with numpy : 0.000644683837890625 nb_pixel_total : 29223 time to create 1 rle with old method : 0.033653974533081055 length of segment : 178 time for calcul the mask position with numpy : 0.0016355514526367188 nb_pixel_total : 65504 time to create 1 rle with old method : 0.07265210151672363 length of segment : 441 time for calcul the mask position with numpy : 0.0002810955047607422 nb_pixel_total : 7268 time to create 1 rle with old method : 0.008263587951660156 length of segment : 124 time for calcul the mask position with numpy : 0.00047659873962402344 nb_pixel_total : 24226 time to create 1 rle with old method : 0.02773737907409668 length of segment : 222 time for calcul the mask position with numpy : 0.00030922889709472656 nb_pixel_total : 11194 time to create 1 rle with old method : 0.012819528579711914 length of segment : 105 time for calcul the mask position with numpy : 0.0008792877197265625 nb_pixel_total : 29524 time to create 1 rle with old method : 0.033188819885253906 length of segment : 283 time for calcul the mask position with numpy : 0.0012650489807128906 nb_pixel_total : 41328 time to create 1 rle with old method : 0.051299095153808594 length of segment : 290 time for calcul the mask position with numpy : 0.0007266998291015625 nb_pixel_total : 15628 time to create 1 rle with old method : 0.019030332565307617 length of segment : 199 time for calcul the mask position with numpy : 0.0034182071685791016 nb_pixel_total : 181911 time to create 1 rle with new method : 0.0148162841796875 length of segment : 372 time for calcul the mask position with numpy : 0.0005538463592529297 nb_pixel_total : 20506 time to create 1 rle with old method : 0.02427196502685547 length of segment : 165 time for calcul the mask position with numpy : 0.00022983551025390625 nb_pixel_total : 7284 time to create 1 rle with old method : 0.009605169296264648 length of segment : 112 time for calcul the mask position with numpy : 0.0003323554992675781 nb_pixel_total : 11968 time to create 1 rle with old method : 0.014573097229003906 length of segment : 159 time for calcul the mask position with numpy : 0.0015878677368164062 nb_pixel_total : 58802 time to create 1 rle with old method : 0.07117772102355957 length of segment : 474 time for calcul the mask position with numpy : 0.0006279945373535156 nb_pixel_total : 18886 time to create 1 rle with old method : 0.022079944610595703 length of segment : 240 time for calcul the mask position with numpy : 0.0011425018310546875 nb_pixel_total : 43250 time to create 1 rle with old method : 0.05328011512756348 length of segment : 282 time for calcul the mask position with numpy : 0.0007250308990478516 nb_pixel_total : 34007 time to create 1 rle with old method : 0.045084238052368164 length of segment : 151 time for calcul the mask position with numpy : 0.001834869384765625 nb_pixel_total : 81030 time to create 1 rle with old method : 0.09912490844726562 length of segment : 303 time for calcul the mask position with numpy : 0.0012774467468261719 nb_pixel_total : 77603 time to create 1 rle with old method : 0.09676814079284668 length of segment : 218 time for calcul the mask position with numpy : 0.0006847381591796875 nb_pixel_total : 35781 time to create 1 rle with old method : 0.041491031646728516 length of segment : 164 time for calcul the mask position with numpy : 0.000766754150390625 nb_pixel_total : 36413 time to create 1 rle with old method : 0.04124021530151367 length of segment : 326 time for calcul the mask position with numpy : 0.0014030933380126953 nb_pixel_total : 60765 time to create 1 rle with old method : 0.06843852996826172 length of segment : 357 time for calcul the mask position with numpy : 0.00020599365234375 nb_pixel_total : 7565 time to create 1 rle with old method : 0.008572101593017578 length of segment : 128 time for calcul the mask position with numpy : 0.00024509429931640625 nb_pixel_total : 10732 time to create 1 rle with old method : 0.012203216552734375 length of segment : 141 time for calcul the mask position with numpy : 0.031215667724609375 nb_pixel_total : 797341 time to create 1 rle with new method : 0.0890800952911377 length of segment : 1659 time for calcul the mask position with numpy : 0.004437446594238281 nb_pixel_total : 194122 time to create 1 rle with new method : 0.01573014259338379 length of segment : 648 time for calcul the mask position with numpy : 0.0013537406921386719 nb_pixel_total : 91033 time to create 1 rle with old method : 0.12328028678894043 length of segment : 247 time for calcul the mask position with numpy : 0.0006051063537597656 nb_pixel_total : 15529 time to create 1 rle with old method : 0.021145343780517578 length of segment : 161 time for calcul the mask position with numpy : 0.0021126270294189453 nb_pixel_total : 97227 time to create 1 rle with old method : 0.13308143615722656 length of segment : 437 time for calcul the mask position with numpy : 0.0013778209686279297 nb_pixel_total : 36377 time to create 1 rle with old method : 0.054299116134643555 length of segment : 262 time for calcul the mask position with numpy : 0.003382444381713867 nb_pixel_total : 74770 time to create 1 rle with old method : 0.08978772163391113 length of segment : 435 time for calcul the mask position with numpy : 0.01113748550415039 nb_pixel_total : 418770 time to create 1 rle with new method : 0.13162922859191895 length of segment : 1336 time for calcul the mask position with numpy : 0.0006878376007080078 nb_pixel_total : 22139 time to create 1 rle with old method : 0.026777029037475586 length of segment : 169 time for calcul the mask position with numpy : 0.0013833045959472656 nb_pixel_total : 56599 time to create 1 rle with old method : 0.1036367416381836 length of segment : 175 time for calcul the mask position with numpy : 0.0012116432189941406 nb_pixel_total : 38372 time to create 1 rle with old method : 0.044816017150878906 length of segment : 317 time for calcul the mask position with numpy : 0.0008747577667236328 nb_pixel_total : 29048 time to create 1 rle with old method : 0.03702402114868164 length of segment : 203 time for calcul the mask position with numpy : 0.00047278404235839844 nb_pixel_total : 15409 time to create 1 rle with old method : 0.018533706665039062 length of segment : 124 time for calcul the mask position with numpy : 0.001787424087524414 nb_pixel_total : 72505 time to create 1 rle with old method : 0.08774018287658691 length of segment : 398 time for calcul the mask position with numpy : 0.0004696846008300781 nb_pixel_total : 11743 time to create 1 rle with old method : 0.014452457427978516 length of segment : 123 time for calcul the mask position with numpy : 0.0020656585693359375 nb_pixel_total : 79626 time to create 1 rle with old method : 0.09259891510009766 length of segment : 378 time for calcul the mask position with numpy : 0.00045490264892578125 nb_pixel_total : 9094 time to create 1 rle with old method : 0.01121211051940918 length of segment : 112 time for calcul the mask position with numpy : 0.0007405281066894531 nb_pixel_total : 26471 time to create 1 rle with old method : 0.03172183036804199 length of segment : 198 time for calcul the mask position with numpy : 0.0010516643524169922 nb_pixel_total : 39371 time to create 1 rle with old method : 0.04640531539916992 length of segment : 213 time for calcul the mask position with numpy : 0.0005440711975097656 nb_pixel_total : 14035 time to create 1 rle with old method : 0.017201662063598633 length of segment : 143 time for calcul the mask position with numpy : 0.0001919269561767578 nb_pixel_total : 8540 time to create 1 rle with old method : 0.010990619659423828 length of segment : 53 time for calcul the mask position with numpy : 0.0004105567932128906 nb_pixel_total : 12557 time to create 1 rle with old method : 0.01663947105407715 length of segment : 145 time for calcul the mask position with numpy : 0.0010750293731689453 nb_pixel_total : 13689 time to create 1 rle with old method : 0.026721954345703125 length of segment : 403 time for calcul the mask position with numpy : 0.001337289810180664 nb_pixel_total : 40527 time to create 1 rle with old method : 0.04796624183654785 length of segment : 250 time for calcul the mask position with numpy : 0.0036706924438476562 nb_pixel_total : 142468 time to create 1 rle with old method : 0.18686938285827637 length of segment : 364 time for calcul the mask position with numpy : 0.007903575897216797 nb_pixel_total : 282175 time to create 1 rle with new method : 0.014976978302001953 length of segment : 664 time for calcul the mask position with numpy : 0.0004620552062988281 nb_pixel_total : 14844 time to create 1 rle with old method : 0.017568111419677734 length of segment : 142 time for calcul the mask position with numpy : 0.0008466243743896484 nb_pixel_total : 27710 time to create 1 rle with old method : 0.03193807601928711 length of segment : 157 time for calcul the mask position with numpy : 0.0018525123596191406 nb_pixel_total : 58175 time to create 1 rle with old method : 0.06800436973571777 length of segment : 273 time for calcul the mask position with numpy : 0.0003380775451660156 nb_pixel_total : 20548 time to create 1 rle with old method : 0.023829221725463867 length of segment : 183 time for calcul the mask position with numpy : 0.000339508056640625 nb_pixel_total : 12911 time to create 1 rle with old method : 0.015539884567260742 length of segment : 114 time for calcul the mask position with numpy : 0.003474712371826172 nb_pixel_total : 144363 time to create 1 rle with old method : 0.16403961181640625 length of segment : 442 time for calcul the mask position with numpy : 0.005673885345458984 nb_pixel_total : 149474 time to create 1 rle with old method : 0.17400431632995605 length of segment : 899 time for calcul the mask position with numpy : 0.0016331672668457031 nb_pixel_total : 51490 time to create 1 rle with old method : 0.05971360206604004 length of segment : 410 time for calcul the mask position with numpy : 0.0016527175903320312 nb_pixel_total : 53507 time to create 1 rle with old method : 0.06238245964050293 length of segment : 357 time for calcul the mask position with numpy : 0.0013589859008789062 nb_pixel_total : 54643 time to create 1 rle with old method : 0.06327629089355469 length of segment : 317 time for calcul the mask position with numpy : 0.0005116462707519531 nb_pixel_total : 12676 time to create 1 rle with old method : 0.015497446060180664 length of segment : 108 time for calcul the mask position with numpy : 0.003285646438598633 nb_pixel_total : 112150 time to create 1 rle with old method : 0.13787627220153809 length of segment : 234 time for calcul the mask position with numpy : 0.0005567073822021484 nb_pixel_total : 16629 time to create 1 rle with old method : 0.02008843421936035 length of segment : 144 time for calcul the mask position with numpy : 0.0032460689544677734 nb_pixel_total : 107319 time to create 1 rle with old method : 0.1271975040435791 length of segment : 474 time for calcul the mask position with numpy : 0.0007102489471435547 nb_pixel_total : 21097 time to create 1 rle with old method : 0.024790525436401367 length of segment : 150 time for calcul the mask position with numpy : 0.0006890296936035156 nb_pixel_total : 27314 time to create 1 rle with old method : 0.032193899154663086 length of segment : 146 time for calcul the mask position with numpy : 0.00022554397583007812 nb_pixel_total : 6176 time to create 1 rle with old method : 0.0074923038482666016 length of segment : 56 time for calcul the mask position with numpy : 0.0030150413513183594 nb_pixel_total : 70985 time to create 1 rle with old method : 0.08083057403564453 length of segment : 472 time for calcul the mask position with numpy : 0.00125885009765625 nb_pixel_total : 52163 time to create 1 rle with old method : 0.05922245979309082 length of segment : 281 time for calcul the mask position with numpy : 0.0003161430358886719 nb_pixel_total : 4910 time to create 1 rle with old method : 0.008364200592041016 length of segment : 104 time for calcul the mask position with numpy : 0.0017383098602294922 nb_pixel_total : 53148 time to create 1 rle with old method : 0.07496070861816406 length of segment : 309 time for calcul the mask position with numpy : 0.006302356719970703 nb_pixel_total : 236424 time to create 1 rle with new method : 0.010048389434814453 length of segment : 812 time for calcul the mask position with numpy : 0.004242420196533203 nb_pixel_total : 168812 time to create 1 rle with new method : 0.005087852478027344 length of segment : 404 time for calcul the mask position with numpy : 0.0063741207122802734 nb_pixel_total : 227719 time to create 1 rle with new method : 0.00973200798034668 length of segment : 1024 time for calcul the mask position with numpy : 0.0008945465087890625 nb_pixel_total : 24850 time to create 1 rle with old method : 0.02972722053527832 length of segment : 160 time for calcul the mask position with numpy : 0.002393960952758789 nb_pixel_total : 61388 time to create 1 rle with old method : 0.07599472999572754 length of segment : 267 time for calcul the mask position with numpy : 0.0007867813110351562 nb_pixel_total : 35486 time to create 1 rle with old method : 0.05460309982299805 length of segment : 393 time for calcul the mask position with numpy : 0.0015268325805664062 nb_pixel_total : 28597 time to create 1 rle with old method : 0.05668973922729492 length of segment : 221 time for calcul the mask position with numpy : 0.003515958786010742 nb_pixel_total : 94158 time to create 1 rle with old method : 0.12681078910827637 length of segment : 392 time for calcul the mask position with numpy : 0.0034210681915283203 nb_pixel_total : 51235 time to create 1 rle with old method : 0.05852937698364258 length of segment : 362 time for calcul the mask position with numpy : 0.0008947849273681641 nb_pixel_total : 26507 time to create 1 rle with old method : 0.030518293380737305 length of segment : 174 time for calcul the mask position with numpy : 0.004970550537109375 nb_pixel_total : 152992 time to create 1 rle with new method : 0.010398387908935547 length of segment : 785 time for calcul the mask position with numpy : 0.0010967254638671875 nb_pixel_total : 40170 time to create 1 rle with old method : 0.04852128028869629 length of segment : 145 time for calcul the mask position with numpy : 0.00020194053649902344 nb_pixel_total : 10140 time to create 1 rle with old method : 0.012135744094848633 length of segment : 135 time for calcul the mask position with numpy : 0.0013136863708496094 nb_pixel_total : 63379 time to create 1 rle with old method : 0.0731198787689209 length of segment : 320 time for calcul the mask position with numpy : 0.0038013458251953125 nb_pixel_total : 60999 time to create 1 rle with old method : 0.07032299041748047 length of segment : 581 time spent for convertir_results : 10.830470323562622 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 243 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 547 save missing photos in datou_result : time spend for datou_step_exec : 62.09439563751221 time spend to save output : 1.6955137252807617 total time spend for step 1 : 63.78990936279297 step2:crop_condition Thu May 22 11:11:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 8 ! batch 1 Loaded 243 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 151 About to insert : list_path_to_insert length 151 new photo from crops ! About to upload 151 photos upload in portfolio : 3736932 init cache_photo without model_param we have 151 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747905135_2512277 we have uploaded 151 photos in the portfolio 3736932 time of upload the photos Elapsed time : 36.57095789909363 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! About to upload 24 photos upload in portfolio : 3736932 init cache_photo without model_param we have 24 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747905180_2512277 we have uploaded 24 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.788658142089844 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3736932 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747905187_2512277 we have uploaded 4 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.5215122699737549 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! Next one ! Next one ! we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 52 About to insert : list_path_to_insert length 52 new photo from crops ! About to upload 52 photos upload in portfolio : 3736932 init cache_photo without model_param we have 52 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747905208_2512277 we have uploaded 52 photos in the portfolio 3736932 time of upload the photos Elapsed time : 17.199713468551636 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747905226_2512277 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9263439178466797 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1747905228_2512277 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.9279406070709229 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1360423397, 1360423392, 1360423153, 1360423136, 1360423098, 1360422967, 1360422951, 1360422948] Looping around the photos to save general results len do output : 235 /1360515220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515240Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515278Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515283Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515583Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515590Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515605Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515634Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515804Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1360515874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423397', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423392', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423153', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423136', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423098', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422967', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422951', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422948', None, None, None, None, None, '2935231') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 713 time used for this insertion : 0.3940436840057373 save_final save missing photos in datou_result : time spend for datou_step_exec : 136.41642260551453 time spend to save output : 0.3990335464477539 total time spend for step 2 : 136.81545615196228 step3:rle_unique_nms_with_priority Thu May 22 11:13:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 243 chid ids of type : 3594 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 6.975055694580078 time for calcul the mask position with numpy : 0.022010087966918945 nb_pixel_total : 21408 time to create 1 rle with old method : 0.025119543075561523 time for calcul the mask position with numpy : 0.021376609802246094 nb_pixel_total : 10751 time to create 1 rle with old method : 0.012603759765625 time for calcul the mask position with numpy : 0.02120232582092285 nb_pixel_total : 9502 time to create 1 rle with old method : 0.011129379272460938 time for calcul the mask position with numpy : 0.021886110305786133 nb_pixel_total : 10369 time to create 1 rle with old method : 0.012145042419433594 time for calcul the mask position with numpy : 0.024208545684814453 nb_pixel_total : 261717 time to create 1 rle with new method : 0.7530381679534912 time for calcul the mask position with numpy : 0.021643400192260742 nb_pixel_total : 22290 time to create 1 rle with old method : 0.025137901306152344 time for calcul the mask position with numpy : 0.03242087364196777 nb_pixel_total : 26368 time to create 1 rle with old method : 0.02892589569091797 time for calcul the mask position with numpy : 0.030879497528076172 nb_pixel_total : 23234 time to create 1 rle with old method : 0.02586221694946289 time for calcul the mask position with numpy : 0.02137303352355957 nb_pixel_total : 119426 time to create 1 rle with old method : 0.13150644302368164 time for calcul the mask position with numpy : 0.021703243255615234 nb_pixel_total : 13503 time to create 1 rle with old method : 0.01500391960144043 time for calcul the mask position with numpy : 0.40484118461608887 nb_pixel_total : 6531672 time to create 1 rle with new method : 0.6674959659576416 create new chi : 2.4412736892700195 time to delete rle : 0.018041133880615234 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.08976101875305176 nb_obj : 20 nb_hashtags : 4 time to prepare the origin masks : 7.615448713302612 time for calcul the mask position with numpy : 0.6237616539001465 nb_pixel_total : 6006917 time to create 1 rle with new method : 0.8467762470245361 time for calcul the mask position with numpy : 0.026990175247192383 nb_pixel_total : 17119 time to create 1 rle with old method : 0.0187375545501709 time for calcul the mask position with numpy : 0.028246402740478516 nb_pixel_total : 31074 time to create 1 rle with old method : 0.05121922492980957 time for calcul the mask position with numpy : 0.03219413757324219 nb_pixel_total : 37167 time to create 1 rle with old method : 0.04355454444885254 time for calcul the mask position with numpy : 0.028608083724975586 nb_pixel_total : 11627 time to create 1 rle with old method : 0.013428926467895508 time for calcul the mask position with numpy : 0.028518199920654297 nb_pixel_total : 45107 time to create 1 rle with old method : 0.051106929779052734 time for calcul the mask position with numpy : 0.02743387222290039 nb_pixel_total : 65501 time to create 1 rle with old method : 0.07354140281677246 time for calcul the mask position with numpy : 0.02831864356994629 nb_pixel_total : 28933 time to create 1 rle with old method : 0.03312802314758301 time for calcul the mask position with numpy : 0.028380870819091797 nb_pixel_total : 55141 time to create 1 rle with old method : 0.062442779541015625 time for calcul the mask position with numpy : 0.027634859085083008 nb_pixel_total : 16986 time to create 1 rle with old method : 0.0186007022857666 time for calcul the mask position with numpy : 0.02799391746520996 nb_pixel_total : 12059 time to create 1 rle with old method : 0.01385641098022461 time for calcul the mask position with numpy : 0.029764652252197266 nb_pixel_total : 281376 time to create 1 rle with new method : 0.5956735610961914 time for calcul the mask position with numpy : 0.028568744659423828 nb_pixel_total : 29224 time to create 1 rle with old method : 0.033675193786621094 time for calcul the mask position with numpy : 0.028438806533813477 nb_pixel_total : 19642 time to create 1 rle with old method : 0.022279739379882812 time for calcul the mask position with numpy : 0.02817821502685547 nb_pixel_total : 11196 time to create 1 rle with old method : 0.012678146362304688 time for calcul the mask position with numpy : 0.029701948165893555 nb_pixel_total : 146494 time to create 1 rle with old method : 0.17010211944580078 time for calcul the mask position with numpy : 0.029174327850341797 nb_pixel_total : 49652 time to create 1 rle with old method : 0.056258440017700195 time for calcul the mask position with numpy : 0.02815532684326172 nb_pixel_total : 101532 time to create 1 rle with old method : 0.11288905143737793 time for calcul the mask position with numpy : 0.027576446533203125 nb_pixel_total : 67399 time to create 1 rle with old method : 0.07467150688171387 time for calcul the mask position with numpy : 0.027585983276367188 nb_pixel_total : 16094 time to create 1 rle with old method : 0.01811385154724121 create new chi : 3.571359395980835 time to delete rle : 0.003019094467163086 batch 1 Loaded 39 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.11598730087280273 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 7.415087699890137 time for calcul the mask position with numpy : 0.5627870559692383 nb_pixel_total : 6579515 time to create 1 rle with new method : 0.7701756954193115 time for calcul the mask position with numpy : 0.02741527557373047 nb_pixel_total : 7288 time to create 1 rle with old method : 0.008141517639160156 time for calcul the mask position with numpy : 0.02833247184753418 nb_pixel_total : 11189 time to create 1 rle with old method : 0.012934207916259766 time for calcul the mask position with numpy : 0.028456687927246094 nb_pixel_total : 18901 time to create 1 rle with old method : 0.021815061569213867 time for calcul the mask position with numpy : 0.03029608726501465 nb_pixel_total : 24231 time to create 1 rle with old method : 0.028312206268310547 time for calcul the mask position with numpy : 0.02846837043762207 nb_pixel_total : 20512 time to create 1 rle with old method : 0.023111820220947266 time for calcul the mask position with numpy : 0.028328657150268555 nb_pixel_total : 7269 time to create 1 rle with old method : 0.00833582878112793 time for calcul the mask position with numpy : 0.02857804298400879 nb_pixel_total : 58795 time to create 1 rle with old method : 0.06612110137939453 time for calcul the mask position with numpy : 0.028229236602783203 nb_pixel_total : 11968 time to create 1 rle with old method : 0.01375436782836914 time for calcul the mask position with numpy : 0.028830528259277344 nb_pixel_total : 41345 time to create 1 rle with old method : 0.04733920097351074 time for calcul the mask position with numpy : 0.028549909591674805 nb_pixel_total : 43314 time to create 1 rle with old method : 0.04899144172668457 time for calcul the mask position with numpy : 0.02798771858215332 nb_pixel_total : 15635 time to create 1 rle with old method : 0.017670631408691406 time for calcul the mask position with numpy : 0.02796173095703125 nb_pixel_total : 29500 time to create 1 rle with old method : 0.03341031074523926 time for calcul the mask position with numpy : 0.02840566635131836 nb_pixel_total : 180778 time to create 1 rle with new method : 0.5158307552337646 create new chi : 2.6305930614471436 time to delete rle : 0.0012209415435791016 batch 1 Loaded 28 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.11519908905029297 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 7.777076482772827 time for calcul the mask position with numpy : 0.4589345455169678 nb_pixel_total : 5438408 time to create 1 rle with new method : 0.6861307621002197 time for calcul the mask position with numpy : 0.029009103775024414 nb_pixel_total : 77611 time to create 1 rle with old method : 0.08700346946716309 time for calcul the mask position with numpy : 0.031804800033569336 nb_pixel_total : 796088 time to create 1 rle with new method : 1.79762864112854 time for calcul the mask position with numpy : 0.02874755859375 nb_pixel_total : 34012 time to create 1 rle with old method : 0.03870868682861328 time for calcul the mask position with numpy : 0.028573274612426758 nb_pixel_total : 36403 time to create 1 rle with old method : 0.04099917411804199 time for calcul the mask position with numpy : 0.028428077697753906 nb_pixel_total : 15527 time to create 1 rle with old method : 0.017608642578125 time for calcul the mask position with numpy : 0.028371334075927734 nb_pixel_total : 7564 time to create 1 rle with old method : 0.00855708122253418 time for calcul the mask position with numpy : 0.02835249900817871 nb_pixel_total : 157226 time to create 1 rle with new method : 0.4728415012359619 time for calcul the mask position with numpy : 0.02800774574279785 nb_pixel_total : 36370 time to create 1 rle with old method : 0.041405677795410156 time for calcul the mask position with numpy : 0.028783321380615234 nb_pixel_total : 81045 time to create 1 rle with old method : 0.09136366844177246 time for calcul the mask position with numpy : 0.02936697006225586 nb_pixel_total : 74781 time to create 1 rle with old method : 0.08419656753540039 time for calcul the mask position with numpy : 0.028319597244262695 nb_pixel_total : 10732 time to create 1 rle with old method : 0.012165069580078125 time for calcul the mask position with numpy : 0.028356552124023438 nb_pixel_total : 97201 time to create 1 rle with old method : 0.11574101448059082 time for calcul the mask position with numpy : 0.033040761947631836 nb_pixel_total : 35785 time to create 1 rle with old method : 0.05097007751464844 time for calcul the mask position with numpy : 0.028398513793945312 nb_pixel_total : 91046 time to create 1 rle with old method : 0.10088133811950684 time for calcul the mask position with numpy : 0.027402639389038086 nb_pixel_total : 60091 time to create 1 rle with old method : 0.06657576560974121 time for calcul the mask position with numpy : 0.027715206146240234 nb_pixel_total : 350 time to create 1 rle with old method : 0.0005848407745361328 create new chi : 4.746288061141968 time to delete rle : 0.0020859241485595703 batch 1 Loaded 34 chid ids of type : 3594 +++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.1185750961303711 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 7.412923336029053 time for calcul the mask position with numpy : 0.5018661022186279 nb_pixel_total : 6186135 time to create 1 rle with new method : 0.40816783905029297 time for calcul the mask position with numpy : 0.0290982723236084 nb_pixel_total : 38368 time to create 1 rle with old method : 0.04438304901123047 time for calcul the mask position with numpy : 0.02905416488647461 nb_pixel_total : 14035 time to create 1 rle with old method : 0.016525745391845703 time for calcul the mask position with numpy : 0.0293731689453125 nb_pixel_total : 79603 time to create 1 rle with old method : 0.09269046783447266 time for calcul the mask position with numpy : 0.028304100036621094 nb_pixel_total : 11744 time to create 1 rle with old method : 0.01356649398803711 time for calcul the mask position with numpy : 0.02788090705871582 nb_pixel_total : 12557 time to create 1 rle with old method : 0.014105558395385742 time for calcul the mask position with numpy : 0.028667926788330078 nb_pixel_total : 22138 time to create 1 rle with old method : 0.025387287139892578 time for calcul the mask position with numpy : 0.02862381935119629 nb_pixel_total : 56442 time to create 1 rle with old method : 0.06447863578796387 time for calcul the mask position with numpy : 0.028462886810302734 nb_pixel_total : 29048 time to create 1 rle with old method : 0.03335976600646973 time for calcul the mask position with numpy : 0.029964208602905273 nb_pixel_total : 418731 time to create 1 rle with new method : 0.7530622482299805 time for calcul the mask position with numpy : 0.029053211212158203 nb_pixel_total : 39369 time to create 1 rle with old method : 0.04497647285461426 time for calcul the mask position with numpy : 0.027959585189819336 nb_pixel_total : 26490 time to create 1 rle with old method : 0.02948284149169922 time for calcul the mask position with numpy : 0.027541637420654297 nb_pixel_total : 10067 time to create 1 rle with old method : 0.011314630508422852 time for calcul the mask position with numpy : 0.028068065643310547 nb_pixel_total : 15410 time to create 1 rle with old method : 0.01765131950378418 time for calcul the mask position with numpy : 0.028910160064697266 nb_pixel_total : 72470 time to create 1 rle with old method : 0.08241033554077148 time for calcul the mask position with numpy : 0.028038978576660156 nb_pixel_total : 8539 time to create 1 rle with old method : 0.009774446487426758 time for calcul the mask position with numpy : 0.02804112434387207 nb_pixel_total : 9094 time to create 1 rle with old method : 0.010435104370117188 create new chi : 2.714374542236328 time to delete rle : 0.0023033618927001953 batch 1 Loaded 34 chid ids of type : 3594 ++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.13848876953125 nb_obj : 15 nb_hashtags : 4 time to prepare the origin masks : 7.455905914306641 time for calcul the mask position with numpy : 0.40035414695739746 nb_pixel_total : 5988281 time to create 1 rle with new method : 0.7593879699707031 time for calcul the mask position with numpy : 0.027671337127685547 nb_pixel_total : 40530 time to create 1 rle with old method : 0.04512596130371094 time for calcul the mask position with numpy : 0.028142690658569336 nb_pixel_total : 14843 time to create 1 rle with old method : 0.01682138442993164 time for calcul the mask position with numpy : 0.027962684631347656 nb_pixel_total : 12902 time to create 1 rle with old method : 0.014570236206054688 time for calcul the mask position with numpy : 0.027761220932006836 nb_pixel_total : 142469 time to create 1 rle with old method : 0.16297411918640137 time for calcul the mask position with numpy : 0.02995133399963379 nb_pixel_total : 282172 time to create 1 rle with new method : 0.5848305225372314 time for calcul the mask position with numpy : 0.02830362319946289 nb_pixel_total : 20336 time to create 1 rle with old method : 0.02312445640563965 time for calcul the mask position with numpy : 0.028164148330688477 nb_pixel_total : 51295 time to create 1 rle with old method : 0.059456825256347656 time for calcul the mask position with numpy : 0.02930450439453125 nb_pixel_total : 27708 time to create 1 rle with old method : 0.03192400932312012 time for calcul the mask position with numpy : 0.0286405086517334 nb_pixel_total : 149496 time to create 1 rle with old method : 0.16930198669433594 time for calcul the mask position with numpy : 0.029428720474243164 nb_pixel_total : 53495 time to create 1 rle with old method : 0.07098269462585449 time for calcul the mask position with numpy : 0.029083728790283203 nb_pixel_total : 144371 time to create 1 rle with old method : 0.1636040210723877 time for calcul the mask position with numpy : 0.02828359603881836 nb_pixel_total : 51488 time to create 1 rle with old method : 0.058252811431884766 time for calcul the mask position with numpy : 0.028429269790649414 nb_pixel_total : 58178 time to create 1 rle with old method : 0.06659793853759766 time for calcul the mask position with numpy : 0.028115034103393555 nb_pixel_total : 12676 time to create 1 rle with old method : 0.014211893081665039 create new chi : 3.1304006576538086 time to delete rle : 0.0017743110656738281 batch 1 Loaded 29 chid ids of type : 3594 +++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.1018683910369873 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 8.563701629638672 time for calcul the mask position with numpy : 0.021847963333129883 nb_pixel_total : 53141 time to create 1 rle with old method : 0.06033635139465332 time for calcul the mask position with numpy : 0.021811962127685547 nb_pixel_total : 4911 time to create 1 rle with old method : 0.005675792694091797 time for calcul the mask position with numpy : 0.02059483528137207 nb_pixel_total : 261 time to create 1 rle with old method : 0.0005197525024414062 time for calcul the mask position with numpy : 0.022095918655395508 nb_pixel_total : 51900 time to create 1 rle with old method : 0.05943775177001953 time for calcul the mask position with numpy : 0.02131962776184082 nb_pixel_total : 70970 time to create 1 rle with old method : 0.08016490936279297 time for calcul the mask position with numpy : 0.022175073623657227 nb_pixel_total : 6175 time to create 1 rle with old method : 0.007318019866943359 time for calcul the mask position with numpy : 0.02124762535095215 nb_pixel_total : 27313 time to create 1 rle with old method : 0.0314335823059082 time for calcul the mask position with numpy : 0.021497011184692383 nb_pixel_total : 21101 time to create 1 rle with old method : 0.023988962173461914 time for calcul the mask position with numpy : 0.02277207374572754 nb_pixel_total : 107315 time to create 1 rle with old method : 0.12062311172485352 time for calcul the mask position with numpy : 0.021313905715942383 nb_pixel_total : 16629 time to create 1 rle with old method : 0.019028902053833008 time for calcul the mask position with numpy : 0.02232837677001953 nb_pixel_total : 112154 time to create 1 rle with old method : 0.12795090675354004 time for calcul the mask position with numpy : 0.5126683712005615 nb_pixel_total : 6578370 time to create 1 rle with new method : 0.4833402633666992 create new chi : 1.8318724632263184 time to delete rle : 0.0008676052093505859 batch 1 Loaded 24 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.1181938648223877 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 7.589620113372803 time for calcul the mask position with numpy : 0.5740420818328857 nb_pixel_total : 5817387 time to create 1 rle with new method : 0.796349048614502 time for calcul the mask position with numpy : 0.027189016342163086 nb_pixel_total : 94136 time to create 1 rle with old method : 0.1018974781036377 time for calcul the mask position with numpy : 0.027385711669921875 nb_pixel_total : 51218 time to create 1 rle with old method : 0.05780625343322754 time for calcul the mask position with numpy : 0.027755260467529297 nb_pixel_total : 61386 time to create 1 rle with old method : 0.0678246021270752 time for calcul the mask position with numpy : 0.02864980697631836 nb_pixel_total : 168816 time to create 1 rle with new method : 0.5645780563354492 time for calcul the mask position with numpy : 0.026956558227539062 nb_pixel_total : 28583 time to create 1 rle with old method : 0.03171229362487793 time for calcul the mask position with numpy : 0.027270793914794922 nb_pixel_total : 10141 time to create 1 rle with old method : 0.011364459991455078 time for calcul the mask position with numpy : 0.027871131896972656 nb_pixel_total : 40174 time to create 1 rle with old method : 0.044840335845947266 time for calcul the mask position with numpy : 0.02863311767578125 nb_pixel_total : 227704 time to create 1 rle with new method : 0.5305101871490479 time for calcul the mask position with numpy : 0.027080059051513672 nb_pixel_total : 26512 time to create 1 rle with old method : 0.02896261215209961 time for calcul the mask position with numpy : 0.027548551559448242 nb_pixel_total : 152984 time to create 1 rle with new method : 0.8672373294830322 time for calcul the mask position with numpy : 0.027592182159423828 nb_pixel_total : 60688 time to create 1 rle with old method : 0.0674734115600586 time for calcul the mask position with numpy : 0.03086566925048828 nb_pixel_total : 235380 time to create 1 rle with new method : 0.5044219493865967 time for calcul the mask position with numpy : 0.027579307556152344 nb_pixel_total : 1363 time to create 1 rle with old method : 0.0020978450775146484 time for calcul the mask position with numpy : 0.028122663497924805 nb_pixel_total : 5763 time to create 1 rle with old method : 0.0066339969635009766 time for calcul the mask position with numpy : 0.027239322662353516 nb_pixel_total : 24851 time to create 1 rle with old method : 0.02691793441772461 time for calcul the mask position with numpy : 0.02695941925048828 nb_pixel_total : 43154 time to create 1 rle with old method : 0.04634666442871094 create new chi : 4.9228222370147705 time to delete rle : 0.0026569366455078125 batch 1 Loaded 34 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.16575407981872559 map_output_result : {1360423397: (0.0, 'Should be the crop_list due to order', 0), 1360423392: (0.0, 'Should be the crop_list due to order', 0), 1360423153: (0.0, 'Should be the crop_list due to order', 0), 1360423136: (0.0, 'Should be the crop_list due to order', 0), 1360423098: (0.0, 'Should be the crop_list due to order', 0), 1360422967: (0.0, 'Should be the crop_list due to order', 0), 1360422951: (0.0, 'Should be the crop_list due to order', 0), 1360422948: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1360423397, 1360423392, 1360423153, 1360423136, 1360423098, 1360422967, 1360422951, 1360422948] Looping around the photos to save general results len do output : 8 /1360423397.Didn't retrieve data . /1360423392.Didn't retrieve data . /1360423153.Didn't retrieve data . /1360423136.Didn't retrieve data . /1360423098.Didn't retrieve data . /1360422967.Didn't retrieve data . /1360422951.Didn't retrieve data . /1360422948.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423397', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423392', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423153', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423136', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423098', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422967', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422951', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422948', None, None, None, None, None, '2935231') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.04602313041687012 save_final save missing photos in datou_result : time spend for datou_step_exec : 89.02993202209473 time spend to save output : 0.046459197998046875 total time spend for step 3 : 89.07639122009277 step4:ventilate_hashtags_in_portfolio Thu May 22 11:15:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 23263226 get user id for portfolio 23263226 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23263226 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','metal','pet_fonce','pet_clair','mal_croppe','background','pehd','environnement','carton','papier','autre')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23263226 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','metal','pet_fonce','pet_clair','mal_croppe','background','pehd','environnement','carton','papier','autre')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23263226 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('flou','metal','pet_fonce','pet_clair','mal_croppe','background','pehd','environnement','carton','papier','autre')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/23263701,23263702,23263703,23263704,23263705,23263706,23263707,23263708,23263709,23263710,23263711?tags=flou,metal,pet_fonce,pet_clair,mal_croppe,background,pehd,environnement,carton,papier,autre Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1360423397, 1360423392, 1360423153, 1360423136, 1360423098, 1360422967, 1360422951, 1360422948] Looping around the photos to save general results len do output : 1 /23263226. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423397', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423392', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423153', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423136', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423098', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422967', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422951', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422948', None, None, None, None, None, '2935231') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.03794264793395996 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.627166986465454 time spend to save output : 0.03819870948791504 total time spend for step 4 : 1.6653656959533691 step5:final Thu May 22 11:15:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1360423397: ('0.12898913724922847',), 1360423392: ('0.12898913724922847',), 1360423153: ('0.12898913724922847',), 1360423136: ('0.12898913724922847',), 1360423098: ('0.12898913724922847',), 1360422967: ('0.12898913724922847',), 1360422951: ('0.12898913724922847',), 1360422948: ('0.12898913724922847',)} new output for save of step final : {1360423397: ('0.12898913724922847',), 1360423392: ('0.12898913724922847',), 1360423153: ('0.12898913724922847',), 1360423136: ('0.12898913724922847',), 1360423098: ('0.12898913724922847',), 1360422967: ('0.12898913724922847',), 1360422951: ('0.12898913724922847',), 1360422948: ('0.12898913724922847',)} [1360423397, 1360423392, 1360423153, 1360423136, 1360423098, 1360422967, 1360422951, 1360422948] Looping around the photos to save general results len do output : 8 /1360423397.Didn't retrieve data . /1360423392.Didn't retrieve data . /1360423153.Didn't retrieve data . /1360423136.Didn't retrieve data . /1360423098.Didn't retrieve data . /1360422967.Didn't retrieve data . /1360422951.Didn't retrieve data . /1360422948.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423397', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423392', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423153', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423136', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423098', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422967', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422951', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422948', None, None, None, None, None, '2935231') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 24 time used for this insertion : 0.03130292892456055 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11906552314758301 time spend to save output : 0.03174138069152832 total time spend for step 5 : 0.15080690383911133 step6:blur_detection Thu May 22 11:15:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 8 time used for this insertion : 0.009834766387939453 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 8 time used for this insertion : 0.009037494659423828 save missing photos in datou_result : time spend for datou_step_exec : 0.021636247634887695 time spend to save output : 0.023422718048095703 total time spend for step 6 : 0.0450589656829834 step7:brightness Thu May 22 11:15:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 8 time used for this insertion : 0.009890079498291016 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 8 time used for this insertion : 0.00969696044921875 save missing photos in datou_result : time spend for datou_step_exec : 0.02549004554748535 time spend to save output : 0.024308443069458008 total time spend for step 7 : 0.04979848861694336 step8:velours_tree Thu May 22 11:15:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.08100342750549316 time spend to save output : 3.4809112548828125e-05 total time spend for step 8 : 0.08103823661804199 step9:send_mail_cod Thu May 22 11:15:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P23263226_22-05-2025_11_15_21.pdf 23263701 imagette232637011747905321 23263702 change filename to text .change filename to text .change filename to text .change filename to text .imagette232637021747905321 23263703 change filename to text .change filename to text .imagette232637031747905321 23263704 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232637041747905321 23263705 imagette232637051747905322 23263706 imagette232637061747905322 23263707 imagette232637071747905322 23263709 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232637091747905322 23263710 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette232637101747905324 23263711 change filename to text .change filename to text .imagette232637111747905325 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=23263226 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/23263701,23263702,23263703,23263704,23263705,23263706,23263707,23263708,23263709,23263710,23263711?tags=flou,metal,pet_fonce,pet_clair,mal_croppe,background,pehd,environnement,carton,papier,autre args[1360423397] : ((1360423397, -1.3838006076275986, 492688767), (1360423397, 0.4444111841352276, 2107752395), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360423392] : ((1360423392, -1.9875520126048989, 492688767), (1360423392, -0.15025951507077073, 496442774), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360423153] : ((1360423153, 0.9260334973280148, 492688767), (1360423153, 0.2696384699567463, 2107752395), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360423136] : ((1360423136, 0.9011555164460692, 492688767), (1360423136, 0.029496194244601148, 2107752395), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360423098] : ((1360423098, -1.7519423000111933, 492688767), (1360423098, 0.031554089798859965, 2107752395), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360422967] : ((1360422967, 0.039547325506745445, 492688767), (1360422967, 0.09669558916689742, 2107752395), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360422951] : ((1360422951, -0.5883789473927161, 492688767), (1360422951, -0.05325281631301003, 2107752395), '0.12898913724922847') We are sending mail with results at report@fotonower.com args[1360422948] : ((1360422948, 0.7451346173218512, 492688767), (1360422948, -0.1828068311997238, 496442774), '0.12898913724922847') We are sending mail with results at report@fotonower.com refus_total : 0.12898913724922847 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=23263226 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23263226_22-05-2025_11_15_21.pdf results_Auto_P23263226_22-05-2025_11_15_21.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23263226_22-05-2025_11_15_21.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3318','23263226','results_Auto_P23263226_22-05-2025_11_15_21.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23263226_22-05-2025_11_15_21.pdf','pdf','','0.4','0.12898913724922847') message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/23263226

https://www.fotonower.com/image?json=false&list_photos_id=1360423397
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360423392
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360423153
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360423136
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360423098
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360422967
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360422951
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1360422948
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 12.90%
Veuillez trouver les photos des contaminants.

exemples de contaminants: metal: https://www.fotonower.com/view/23263702?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/23263703?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/23263704?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/23263709?limit=200
exemples de contaminants: papier: https://www.fotonower.com/view/23263710?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/23263711?limit=200
Veuillez trouver le rapport en pdf:https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23263226_22-05-2025_11_15_21.pdf.

Lien vers velours :https://www.fotonower.com/velours/23263701,23263702,23263703,23263704,23263705,23263706,23263707,23263708,23263709,23263710,23263711?tags=flou,metal,pet_fonce,pet_clair,mal_croppe,background,pehd,environnement,carton,papier,autre.


L'équipe Fotonower 202 b'' Server: nginx Date: Thu, 22 May 2025 09:15:27 GMT Content-Length: 0 Connection: close X-Message-Id: dF-SV_jeRB-pIC3fgOd-Bw Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1360423397, 1360423392, 1360423153, 1360423136, 1360423098, 1360422967, 1360422951, 1360422948] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423397', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423392', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423153', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423136', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423098', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422967', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422951', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422948', None, None, None, None, None, '2935231') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 8 time used for this insertion : 0.03748798370361328 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.903054714202881 time spend to save output : 0.037763357162475586 total time spend for step 9 : 6.9408180713653564 step10:split_time_score Thu May 22 11:15:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('07', 8),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 22052025 23263226 Nombre de photos uploadées : 8 / 23040 (0%) 22052025 23263226 Nombre de photos taguées (types de déchets): 0 / 8 (0%) 22052025 23263226 Nombre de photos taguées (volume) : 0 / 8 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ???????? elapsed_time : fill_and_build_computed_from_old_data 0.0004794597625732422 elapsed_time : insert_dashboard_record_day_entry 0.03435397148132324 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.12898913724922847 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P23263226_22-05-2025_11_15_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 23263226 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23263226 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'22052025': {'nb_upload': 8, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1360423397, 1360423392, 1360423153, 1360423136, 1360423098, 1360422967, 1360422951, 1360422948] Looping around the photos to save general results len do output : 1 /23263226Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423397', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423392', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423153', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423136', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360423098', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422967', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422951', None, None, None, None, None, '2935231') ('3318', None, None, None, None, None, None, None, '2935231') ('3318', '23263226', '1360422948', None, None, None, None, None, '2935231') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.01369786262512207 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.2571985721588135 time spend to save output : 0.013975143432617188 total time spend for step 10 : 0.27117371559143066 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 8 set_done_treatment 158.19user 73.72system 5:03.83elapsed 76%CPU (0avgtext+0avgdata 4897868maxresident)k 503720inputs+110592outputs (13major+11823914minor)pagefaults 0swaps