python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3386087 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 3318, datou_cur_ids : ['2598940'] with mtr_portfolio_ids : ['20669347'] and first list_photo_ids : [] new path : /proc/3386087/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.3859174251556396 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 10 step1:mask_detect Wed Feb 19 09:10:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10151 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-19 09:10:35.139494: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-19 09:10:35.167311: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-19 09:10:35.169200: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f6460000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-19 09:10:35.169290: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-19 09:10:35.172938: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-19 09:10:35.317126: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x24978290 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-19 09:10:35.317225: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-19 09:10:35.318945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-19 09:10:35.319661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-19 09:10:35.323784: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-19 09:10:35.328230: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-19 09:10:35.328704: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-19 09:10:35.332332: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-19 09:10:35.333676: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-19 09:10:35.340141: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-19 09:10:35.341850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-19 09:10:35.341971: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-19 09:10:35.342961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-19 09:10:35.342988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-19 09:10:35.342999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-19 09:10:35.344544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9009 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-19 09:10:35.762142: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-19 09:10:35.762274: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-19 09:10:35.762302: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-19 09:10:35.762323: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-19 09:10:35.762341: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-19 09:10:35.762360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-19 09:10:35.762380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-19 09:10:35.762400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-19 09:10:35.764515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-19 09:10:35.766726: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-19 09:10:35.766828: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-19 09:10:35.766850: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-19 09:10:35.766869: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-19 09:10:35.766887: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-19 09:10:35.766919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-19 09:10:35.766937: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-19 09:10:35.766953: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-19 09:10:35.768269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-19 09:10:35.768319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-19 09:10:35.768329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-19 09:10:35.768338: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-19 09:10:35.769706: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9009 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-19 09:10:48.662037: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-19 09:10:48.966751: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-19 09:10:51.078416: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.079677: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.60G (3865470464 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.080866: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 3.24G (3478923264 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.082124: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3131030784 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.885173: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.885274: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-02-19 09:10:51.955376: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.955444: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:51.956680: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.956705: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:51.964775: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.964814: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:51.965779: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.965801: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:51.972074: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.972105: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:51.972858: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:51.972875: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:52.010353: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.010466: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:52.010485: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-19 09:10:52.011782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.011815: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:52.013086: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.013113: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:52.020932: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.020994: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-19 09:10:52.022237: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.023529: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.024799: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.037436: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.038774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.061169: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.062412: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.063604: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.064772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.070473: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.071689: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.072849: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.073999: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.075925: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.088163: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.089414: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.101647: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.102932: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.104148: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.105348: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.106586: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-19 09:10:52.107823: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 4.00G (4294967296 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 Detection mask done ! Trying to reset tf kernel 3386908 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4663 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5635 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2847 Catched exception ! Connect or reconnect ! thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0009367465972900391 nb_pixel_total : 21764 time to create 1 rle with old method : 0.030029773712158203 length of segment : 190 time for calcul the mask position with numpy : 0.0023131370544433594 nb_pixel_total : 56609 time to create 1 rle with old method : 0.07080674171447754 length of segment : 430 time for calcul the mask position with numpy : 0.0202329158782959 nb_pixel_total : 340323 time to create 1 rle with new method : 0.023877382278442383 length of segment : 300 time for calcul the mask position with numpy : 0.0002560615539550781 nb_pixel_total : 12077 time to create 1 rle with old method : 0.013847112655639648 length of segment : 110 time for calcul the mask position with numpy : 0.0008630752563476562 nb_pixel_total : 40360 time to create 1 rle with old method : 0.045610904693603516 length of segment : 443 time for calcul the mask position with numpy : 0.0006494522094726562 nb_pixel_total : 30196 time to create 1 rle with old method : 0.033948659896850586 length of segment : 230 time for calcul the mask position with numpy : 0.00025844573974609375 nb_pixel_total : 12117 time to create 1 rle with old method : 0.01540994644165039 length of segment : 111 time for calcul the mask position with numpy : 0.0018639564514160156 nb_pixel_total : 130812 time to create 1 rle with old method : 0.15533804893493652 length of segment : 350 time for calcul the mask position with numpy : 0.00027108192443847656 nb_pixel_total : 10586 time to create 1 rle with old method : 0.012186050415039062 length of segment : 139 time for calcul the mask position with numpy : 0.0006968975067138672 nb_pixel_total : 42761 time to create 1 rle with old method : 0.05027961730957031 length of segment : 299 time for calcul the mask position with numpy : 0.0006017684936523438 nb_pixel_total : 24048 time to create 1 rle with old method : 0.02730274200439453 length of segment : 189 time for calcul the mask position with numpy : 0.0024271011352539062 nb_pixel_total : 131992 time to create 1 rle with old method : 0.16488957405090332 length of segment : 539 time for calcul the mask position with numpy : 0.0002040863037109375 nb_pixel_total : 6378 time to create 1 rle with old method : 0.007821321487426758 length of segment : 73 time for calcul the mask position with numpy : 0.0006308555603027344 nb_pixel_total : 26982 time to create 1 rle with old method : 0.03196525573730469 length of segment : 205 time for calcul the mask position with numpy : 0.0005438327789306641 nb_pixel_total : 24284 time to create 1 rle with old method : 0.028170108795166016 length of segment : 179 time for calcul the mask position with numpy : 0.0003421306610107422 nb_pixel_total : 4250 time to create 1 rle with old method : 0.005812168121337891 length of segment : 89 time for calcul the mask position with numpy : 0.00021839141845703125 nb_pixel_total : 6113 time to create 1 rle with old method : 0.013728141784667969 length of segment : 92 time for calcul the mask position with numpy : 0.0003764629364013672 nb_pixel_total : 12687 time to create 1 rle with old method : 0.014635801315307617 length of segment : 164 time for calcul the mask position with numpy : 0.0004611015319824219 nb_pixel_total : 9220 time to create 1 rle with old method : 0.011058568954467773 length of segment : 106 time for calcul the mask position with numpy : 0.001894235610961914 nb_pixel_total : 82913 time to create 1 rle with old method : 0.11472105979919434 length of segment : 509 time for calcul the mask position with numpy : 0.0076258182525634766 nb_pixel_total : 309731 time to create 1 rle with new method : 0.0225069522857666 length of segment : 380 time for calcul the mask position with numpy : 0.002626657485961914 nb_pixel_total : 79737 time to create 1 rle with old method : 0.10186505317687988 length of segment : 408 time for calcul the mask position with numpy : 0.0021317005157470703 nb_pixel_total : 62986 time to create 1 rle with old method : 0.07193636894226074 length of segment : 287 time for calcul the mask position with numpy : 0.003439188003540039 nb_pixel_total : 102125 time to create 1 rle with old method : 0.12474226951599121 length of segment : 432 time for calcul the mask position with numpy : 0.02442455291748047 nb_pixel_total : 416097 time to create 1 rle with new method : 0.10567188262939453 length of segment : 307 time for calcul the mask position with numpy : 0.0007159709930419922 nb_pixel_total : 11611 time to create 1 rle with old method : 0.013864517211914062 length of segment : 191 time for calcul the mask position with numpy : 0.0003108978271484375 nb_pixel_total : 10360 time to create 1 rle with old method : 0.012184858322143555 length of segment : 123 time for calcul the mask position with numpy : 0.0006017684936523438 nb_pixel_total : 11244 time to create 1 rle with old method : 0.013434886932373047 length of segment : 119 time for calcul the mask position with numpy : 0.001863241195678711 nb_pixel_total : 52631 time to create 1 rle with old method : 0.05909371376037598 length of segment : 442 time for calcul the mask position with numpy : 0.014396905899047852 nb_pixel_total : 232932 time to create 1 rle with new method : 0.22149419784545898 length of segment : 878 time for calcul the mask position with numpy : 0.0008041858673095703 nb_pixel_total : 22295 time to create 1 rle with old method : 0.026486873626708984 length of segment : 165 time for calcul the mask position with numpy : 0.0011553764343261719 nb_pixel_total : 25482 time to create 1 rle with old method : 0.02977728843688965 length of segment : 216 time for calcul the mask position with numpy : 0.0010688304901123047 nb_pixel_total : 49986 time to create 1 rle with old method : 0.05954861640930176 length of segment : 330 time for calcul the mask position with numpy : 0.0028748512268066406 nb_pixel_total : 120244 time to create 1 rle with old method : 0.14610505104064941 length of segment : 520 time for calcul the mask position with numpy : 0.0007648468017578125 nb_pixel_total : 17838 time to create 1 rle with old method : 0.020972013473510742 length of segment : 184 time for calcul the mask position with numpy : 0.0003998279571533203 nb_pixel_total : 9098 time to create 1 rle with old method : 0.010949373245239258 length of segment : 67 time for calcul the mask position with numpy : 0.0012693405151367188 nb_pixel_total : 66013 time to create 1 rle with old method : 0.07623553276062012 length of segment : 369 time for calcul the mask position with numpy : 0.0019505023956298828 nb_pixel_total : 78929 time to create 1 rle with old method : 0.08914804458618164 length of segment : 364 time for calcul the mask position with numpy : 0.0029959678649902344 nb_pixel_total : 79900 time to create 1 rle with old method : 0.09179043769836426 length of segment : 546 time for calcul the mask position with numpy : 0.0024814605712890625 nb_pixel_total : 54021 time to create 1 rle with old method : 0.06383514404296875 length of segment : 260 time for calcul the mask position with numpy : 0.0014405250549316406 nb_pixel_total : 40204 time to create 1 rle with old method : 0.04747509956359863 length of segment : 266 time for calcul the mask position with numpy : 0.0019426345825195312 nb_pixel_total : 57276 time to create 1 rle with old method : 0.0668632984161377 length of segment : 437 time for calcul the mask position with numpy : 0.0003218650817871094 nb_pixel_total : 4667 time to create 1 rle with old method : 0.006226539611816406 length of segment : 53 time for calcul the mask position with numpy : 0.0006365776062011719 nb_pixel_total : 11689 time to create 1 rle with old method : 0.014935493469238281 length of segment : 108 time for calcul the mask position with numpy : 0.0008704662322998047 nb_pixel_total : 21850 time to create 1 rle with old method : 0.026388168334960938 length of segment : 204 time for calcul the mask position with numpy : 0.0019447803497314453 nb_pixel_total : 50122 time to create 1 rle with old method : 0.05796337127685547 length of segment : 301 time for calcul the mask position with numpy : 0.000415802001953125 nb_pixel_total : 10819 time to create 1 rle with old method : 0.01294565200805664 length of segment : 117 time for calcul the mask position with numpy : 0.0017163753509521484 nb_pixel_total : 69241 time to create 1 rle with old method : 0.07860565185546875 length of segment : 253 time for calcul the mask position with numpy : 0.006979227066040039 nb_pixel_total : 183889 time to create 1 rle with new method : 0.016497373580932617 length of segment : 804 time for calcul the mask position with numpy : 0.00022077560424804688 nb_pixel_total : 1039 time to create 1 rle with old method : 0.001336812973022461 length of segment : 68 time for calcul the mask position with numpy : 0.0005779266357421875 nb_pixel_total : 29768 time to create 1 rle with old method : 0.03490424156188965 length of segment : 152 time for calcul the mask position with numpy : 0.0016314983367919922 nb_pixel_total : 40801 time to create 1 rle with old method : 0.05829501152038574 length of segment : 432 time for calcul the mask position with numpy : 0.0033876895904541016 nb_pixel_total : 92105 time to create 1 rle with old method : 0.10466313362121582 length of segment : 482 time for calcul the mask position with numpy : 0.01392054557800293 nb_pixel_total : 381301 time to create 1 rle with new method : 0.14932894706726074 length of segment : 346 time for calcul the mask position with numpy : 0.0003273487091064453 nb_pixel_total : 7887 time to create 1 rle with old method : 0.010228157043457031 length of segment : 69 time for calcul the mask position with numpy : 0.00026702880859375 nb_pixel_total : 5715 time to create 1 rle with old method : 0.00736689567565918 length of segment : 72 time for calcul the mask position with numpy : 0.004489421844482422 nb_pixel_total : 113381 time to create 1 rle with old method : 0.14072751998901367 length of segment : 483 time for calcul the mask position with numpy : 0.0005548000335693359 nb_pixel_total : 16600 time to create 1 rle with old method : 0.01904773712158203 length of segment : 141 time for calcul the mask position with numpy : 0.00164031982421875 nb_pixel_total : 107577 time to create 1 rle with old method : 0.12495923042297363 length of segment : 382 time for calcul the mask position with numpy : 0.008196115493774414 nb_pixel_total : 251020 time to create 1 rle with new method : 0.02967667579650879 length of segment : 1245 time for calcul the mask position with numpy : 0.0019412040710449219 nb_pixel_total : 47775 time to create 1 rle with old method : 0.08479142189025879 length of segment : 518 time for calcul the mask position with numpy : 0.0025339126586914062 nb_pixel_total : 103062 time to create 1 rle with old method : 0.13703441619873047 length of segment : 435 time for calcul the mask position with numpy : 0.0015060901641845703 nb_pixel_total : 52990 time to create 1 rle with old method : 0.06822776794433594 length of segment : 337 time for calcul the mask position with numpy : 0.00685429573059082 nb_pixel_total : 253371 time to create 1 rle with new method : 0.016060829162597656 length of segment : 602 time for calcul the mask position with numpy : 0.0006532669067382812 nb_pixel_total : 20384 time to create 1 rle with old method : 0.025385618209838867 length of segment : 184 time for calcul the mask position with numpy : 0.002584218978881836 nb_pixel_total : 107399 time to create 1 rle with old method : 0.12499165534973145 length of segment : 480 time for calcul the mask position with numpy : 0.0009496212005615234 nb_pixel_total : 38417 time to create 1 rle with old method : 0.05823540687561035 length of segment : 157 time for calcul the mask position with numpy : 0.1232767105102539 nb_pixel_total : 417717 time to create 1 rle with new method : 0.09842061996459961 length of segment : 833 time for calcul the mask position with numpy : 0.0012900829315185547 nb_pixel_total : 33757 time to create 1 rle with old method : 0.03881382942199707 length of segment : 245 time for calcul the mask position with numpy : 0.0013518333435058594 nb_pixel_total : 26089 time to create 1 rle with old method : 0.03167152404785156 length of segment : 342 time for calcul the mask position with numpy : 0.0006246566772460938 nb_pixel_total : 21079 time to create 1 rle with old method : 0.024518251419067383 length of segment : 176 time for calcul the mask position with numpy : 0.0002720355987548828 nb_pixel_total : 5058 time to create 1 rle with old method : 0.006007194519042969 length of segment : 100 time for calcul the mask position with numpy : 0.008369207382202148 nb_pixel_total : 360877 time to create 1 rle with new method : 0.05289649963378906 length of segment : 377 time for calcul the mask position with numpy : 0.0005712509155273438 nb_pixel_total : 24505 time to create 1 rle with old method : 0.028848648071289062 length of segment : 187 time for calcul the mask position with numpy : 0.0006182193756103516 nb_pixel_total : 21834 time to create 1 rle with old method : 0.02530694007873535 length of segment : 171 time for calcul the mask position with numpy : 0.0016717910766601562 nb_pixel_total : 21355 time to create 1 rle with old method : 0.025251150131225586 length of segment : 211 time for calcul the mask position with numpy : 0.004715442657470703 nb_pixel_total : 196739 time to create 1 rle with new method : 0.010264873504638672 length of segment : 465 time for calcul the mask position with numpy : 0.010816812515258789 nb_pixel_total : 370317 time to create 1 rle with new method : 0.03908801078796387 length of segment : 302 time for calcul the mask position with numpy : 0.00038123130798339844 nb_pixel_total : 10449 time to create 1 rle with old method : 0.012205362319946289 length of segment : 118 time for calcul the mask position with numpy : 0.0006501674652099609 nb_pixel_total : 27446 time to create 1 rle with old method : 0.03316640853881836 length of segment : 215 time for calcul the mask position with numpy : 0.00029587745666503906 nb_pixel_total : 7621 time to create 1 rle with old method : 0.00907278060913086 length of segment : 130 time for calcul the mask position with numpy : 0.011473894119262695 nb_pixel_total : 191887 time to create 1 rle with new method : 0.03679299354553223 length of segment : 552 time for calcul the mask position with numpy : 0.008271217346191406 nb_pixel_total : 185535 time to create 1 rle with new method : 0.017939329147338867 length of segment : 608 time for calcul the mask position with numpy : 0.0007979869842529297 nb_pixel_total : 16234 time to create 1 rle with old method : 0.01886439323425293 length of segment : 204 time for calcul the mask position with numpy : 0.007410287857055664 nb_pixel_total : 175550 time to create 1 rle with new method : 0.015798091888427734 length of segment : 742 time for calcul the mask position with numpy : 0.002016305923461914 nb_pixel_total : 40089 time to create 1 rle with old method : 0.05008649826049805 length of segment : 379 time for calcul the mask position with numpy : 0.00857996940612793 nb_pixel_total : 188076 time to create 1 rle with new method : 0.014202594757080078 length of segment : 742 time for calcul the mask position with numpy : 0.0006277561187744141 nb_pixel_total : 10648 time to create 1 rle with old method : 0.01233053207397461 length of segment : 163 time for calcul the mask position with numpy : 0.0011990070343017578 nb_pixel_total : 16528 time to create 1 rle with old method : 0.019800424575805664 length of segment : 141 time for calcul the mask position with numpy : 0.0013513565063476562 nb_pixel_total : 46202 time to create 1 rle with old method : 0.05317234992980957 length of segment : 347 time for calcul the mask position with numpy : 0.0006439685821533203 nb_pixel_total : 9214 time to create 1 rle with old method : 0.014113664627075195 length of segment : 107 time for calcul the mask position with numpy : 0.01028585433959961 nb_pixel_total : 226388 time to create 1 rle with new method : 0.02011251449584961 length of segment : 872 time for calcul the mask position with numpy : 0.00691533088684082 nb_pixel_total : 202092 time to create 1 rle with new method : 0.013341188430786133 length of segment : 988 time for calcul the mask position with numpy : 0.003107309341430664 nb_pixel_total : 87629 time to create 1 rle with old method : 0.09996867179870605 length of segment : 191 time for calcul the mask position with numpy : 0.0005285739898681641 nb_pixel_total : 6685 time to create 1 rle with old method : 0.013174057006835938 length of segment : 114 time for calcul the mask position with numpy : 0.0012726783752441406 nb_pixel_total : 10159 time to create 1 rle with old method : 0.019341707229614258 length of segment : 315 time for calcul the mask position with numpy : 0.0017659664154052734 nb_pixel_total : 48929 time to create 1 rle with old method : 0.05572843551635742 length of segment : 304 time for calcul the mask position with numpy : 0.0018923282623291016 nb_pixel_total : 61141 time to create 1 rle with old method : 0.06940293312072754 length of segment : 315 time for calcul the mask position with numpy : 0.003705263137817383 nb_pixel_total : 168513 time to create 1 rle with new method : 0.008846282958984375 length of segment : 510 time for calcul the mask position with numpy : 0.0022192001342773438 nb_pixel_total : 93001 time to create 1 rle with old method : 0.12655854225158691 length of segment : 493 time for calcul the mask position with numpy : 0.0026862621307373047 nb_pixel_total : 73255 time to create 1 rle with old method : 0.0882411003112793 length of segment : 460 time for calcul the mask position with numpy : 0.0023314952850341797 nb_pixel_total : 73536 time to create 1 rle with old method : 0.08770895004272461 length of segment : 285 time spent for convertir_results : 12.27778673171997 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 102 chid ids of type : 3594 Number RLEs to save : 33167 save missing photos in datou_result : time spend for datou_step_exec : 67.55992770195007 time spend to save output : 3.057600498199463 total time spend for step 1 : 70.61752820014954 step2:crop_condition Wed Feb 19 09:11:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3594 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 102 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : papier param for this class : {'min_score': 0.7} filtre for class : papier hashtag_id of this class : 492668766 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 55 About to insert : list_path_to_insert length 55 new photo from crops ! About to upload 55 photos upload in portfolio : 3736932 init cache_photo without model_param we have 55 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739952720_3386087 we have uploaded 55 photos in the portfolio 3736932 time of upload the photos Elapsed time : 13.39334225654602 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.7} filtre for class : carton hashtag_id of this class : 492774966 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! About to upload 22 photos upload in portfolio : 3736932 init cache_photo without model_param we have 22 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739952742_3386087 we have uploaded 22 photos in the portfolio 3736932 time of upload the photos Elapsed time : 6.831213712692261 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.7} filtre for class : metal hashtag_id of this class : 492628673 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3736932 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739952751_3386087 we have uploaded 2 photos in the portfolio 3736932 time of upload the photos Elapsed time : 1.7571802139282227 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.7} filtre for class : pet_clair hashtag_id of this class : 2107755846 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3736932 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739952771_3386087 we have uploaded 21 photos in the portfolio 3736932 time of upload the photos Elapsed time : 5.131325960159302 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.7} filtre for class : autre hashtag_id of this class : 494826614 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739952778_3386087 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.4648780822753906 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.7} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.7} filtre for class : pet_fonce hashtag_id of this class : 2107755900 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3736932 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739952782_3386087 we have uploaded 1 photos in the portfolio 3736932 time of upload the photos Elapsed time : 0.6209592819213867 we have finished the crop for the class : pet_fonce delete rles from all chi we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles we have 0 chi objets contains the rles Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338503848, 1338503842, 1338502347, 1338502244, 1338502242, 1338502132, 1338501699, 1338501695, 1338501694, 1338501691] Looping around the photos to save general results len do output : 102 /1338506035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506117Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506125Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506142Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506143Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506144Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506145Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506146Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506147Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506148Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506149Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506150Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506152Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506158Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506161Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338506167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503848', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503842', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502347', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502244', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502242', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502132', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501699', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501695', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501694', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501691', None, None, None, None, None, '2598940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 316 time used for this insertion : 0.04958176612854004 save_final save missing photos in datou_result : time spend for datou_step_exec : 80.19367003440857 time spend to save output : 0.05369758605957031 total time spend for step 2 : 80.24736762046814 step3:rle_unique_nms_with_priority Wed Feb 19 09:13:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 102 chid ids of type : 3594 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 4.871663570404053 time for calcul the mask position with numpy : 0.6853783130645752 nb_pixel_total : 6352635 time to create 1 rle with new method : 0.6957452297210693 time for calcul the mask position with numpy : 0.02167034149169922 nb_pixel_total : 42761 time to create 1 rle with old method : 0.050045013427734375 time for calcul the mask position with numpy : 0.02166128158569336 nb_pixel_total : 10586 time to create 1 rle with old method : 0.011930227279663086 time for calcul the mask position with numpy : 0.0237276554107666 nb_pixel_total : 130812 time to create 1 rle with old method : 0.14962339401245117 time for calcul the mask position with numpy : 0.020922183990478516 nb_pixel_total : 12117 time to create 1 rle with old method : 0.014389753341674805 time for calcul the mask position with numpy : 0.021126270294189453 nb_pixel_total : 30196 time to create 1 rle with old method : 0.034948110580444336 time for calcul the mask position with numpy : 0.024111509323120117 nb_pixel_total : 40360 time to create 1 rle with old method : 0.04851102828979492 time for calcul the mask position with numpy : 0.020966291427612305 nb_pixel_total : 12077 time to create 1 rle with old method : 0.013851642608642578 time for calcul the mask position with numpy : 0.02390265464782715 nb_pixel_total : 340323 time to create 1 rle with new method : 1.1429193019866943 time for calcul the mask position with numpy : 0.022858142852783203 nb_pixel_total : 56609 time to create 1 rle with old method : 0.0637979507446289 time for calcul the mask position with numpy : 0.021220922470092773 nb_pixel_total : 21764 time to create 1 rle with old method : 0.02428436279296875 create new chi : 3.2141478061676025 time to delete rle : 0.015518426895141602 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++Number RLEs to save : 7364 TO DO : save crop sub photo not yet done ! save time : 0.6395218372344971 nb_obj : 13 nb_hashtags : 3 time to prepare the origin masks : 6.315260648727417 time for calcul the mask position with numpy : 0.6489520072937012 nb_pixel_total : 6269772 time to create 1 rle with new method : 1.426321029663086 time for calcul the mask position with numpy : 0.03417062759399414 nb_pixel_total : 62986 time to create 1 rle with old method : 0.07578754425048828 time for calcul the mask position with numpy : 0.03186917304992676 nb_pixel_total : 79737 time to create 1 rle with old method : 0.11534905433654785 time for calcul the mask position with numpy : 0.03562045097351074 nb_pixel_total : 309731 time to create 1 rle with new method : 1.011075735092163 time for calcul the mask position with numpy : 0.022552967071533203 nb_pixel_total : 82913 time to create 1 rle with old method : 0.09335613250732422 time for calcul the mask position with numpy : 0.02116537094116211 nb_pixel_total : 8367 time to create 1 rle with old method : 0.009977102279663086 time for calcul the mask position with numpy : 0.021502256393432617 nb_pixel_total : 12687 time to create 1 rle with old method : 0.014867782592773438 time for calcul the mask position with numpy : 0.021818876266479492 nb_pixel_total : 6113 time to create 1 rle with old method : 0.00686192512512207 time for calcul the mask position with numpy : 0.021866559982299805 nb_pixel_total : 4250 time to create 1 rle with old method : 0.004991292953491211 time for calcul the mask position with numpy : 0.021446943283081055 nb_pixel_total : 24284 time to create 1 rle with old method : 0.0288088321685791 time for calcul the mask position with numpy : 0.02263355255126953 nb_pixel_total : 26982 time to create 1 rle with old method : 0.03302502632141113 time for calcul the mask position with numpy : 0.03455662727355957 nb_pixel_total : 6378 time to create 1 rle with old method : 0.008019208908081055 time for calcul the mask position with numpy : 0.03470015525817871 nb_pixel_total : 131992 time to create 1 rle with old method : 0.1584157943725586 time for calcul the mask position with numpy : 0.03707170486450195 nb_pixel_total : 24048 time to create 1 rle with old method : 0.02848076820373535 create new chi : 4.0860066413879395 time to delete rle : 0.0013513565063476562 batch 1 Loaded 27 chid ids of type : 3594 +++++++++++++++++++++Number RLEs to save : 8558 TO DO : save crop sub photo not yet done ! save time : 0.5344915390014648 nb_obj : 17 nb_hashtags : 3 time to prepare the origin masks : 12.226490020751953 time for calcul the mask position with numpy : 0.8304541110992432 nb_pixel_total : 5755292 time to create 1 rle with new method : 1.0432722568511963 time for calcul the mask position with numpy : 0.051906585693359375 nb_pixel_total : 54021 time to create 1 rle with old method : 0.0667414665222168 time for calcul the mask position with numpy : 0.03940844535827637 nb_pixel_total : 72550 time to create 1 rle with old method : 0.08515501022338867 time for calcul the mask position with numpy : 0.03777933120727539 nb_pixel_total : 78779 time to create 1 rle with old method : 0.08850479125976562 time for calcul the mask position with numpy : 0.03498196601867676 nb_pixel_total : 7655 time to create 1 rle with old method : 0.009624719619750977 time for calcul the mask position with numpy : 0.035648345947265625 nb_pixel_total : 9098 time to create 1 rle with old method : 0.010709762573242188 time for calcul the mask position with numpy : 0.03658342361450195 nb_pixel_total : 17838 time to create 1 rle with old method : 0.020639657974243164 time for calcul the mask position with numpy : 0.037154436111450195 nb_pixel_total : 120244 time to create 1 rle with old method : 0.13436508178710938 time for calcul the mask position with numpy : 0.035456180572509766 nb_pixel_total : 49986 time to create 1 rle with old method : 0.05711197853088379 time for calcul the mask position with numpy : 0.03458142280578613 nb_pixel_total : 25482 time to create 1 rle with old method : 0.028608322143554688 time for calcul the mask position with numpy : 0.034470558166503906 nb_pixel_total : 22295 time to create 1 rle with old method : 0.026169538497924805 time for calcul the mask position with numpy : 0.0394597053527832 nb_pixel_total : 232932 time to create 1 rle with new method : 0.6856186389923096 time for calcul the mask position with numpy : 0.038374900817871094 nb_pixel_total : 52631 time to create 1 rle with old method : 0.06139659881591797 time for calcul the mask position with numpy : 0.03673720359802246 nb_pixel_total : 11244 time to create 1 rle with old method : 0.012776613235473633 time for calcul the mask position with numpy : 0.03636646270751953 nb_pixel_total : 10360 time to create 1 rle with old method : 0.011462926864624023 time for calcul the mask position with numpy : 0.03637242317199707 nb_pixel_total : 11611 time to create 1 rle with old method : 0.012955665588378906 time for calcul the mask position with numpy : 0.03909444808959961 nb_pixel_total : 416097 time to create 1 rle with new method : 0.7571403980255127 time for calcul the mask position with numpy : 0.03696799278259277 nb_pixel_total : 102125 time to create 1 rle with old method : 0.11704611778259277 create new chi : 4.795785665512085 time to delete rle : 0.0019779205322265625 batch 1 Loaded 35 chid ids of type : 3594 +++++++++++++++++++++++Number RLEs to save : 12617 TO DO : save crop sub photo not yet done ! save time : 0.7379515171051025 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 5.97498345375061 time for calcul the mask position with numpy : 0.7302112579345703 nb_pixel_total : 6570554 time to create 1 rle with new method : 0.6018517017364502 time for calcul the mask position with numpy : 0.03329920768737793 nb_pixel_total : 29768 time to create 1 rle with old method : 0.033859968185424805 time for calcul the mask position with numpy : 0.03463482856750488 nb_pixel_total : 1039 time to create 1 rle with old method : 0.001260519027709961 time for calcul the mask position with numpy : 0.03631258010864258 nb_pixel_total : 183011 time to create 1 rle with new method : 0.5016987323760986 time for calcul the mask position with numpy : 0.03564047813415527 nb_pixel_total : 69241 time to create 1 rle with old method : 0.10358381271362305 time for calcul the mask position with numpy : 0.037435054779052734 nb_pixel_total : 10819 time to create 1 rle with old method : 0.012981653213500977 time for calcul the mask position with numpy : 0.035628557205200195 nb_pixel_total : 50122 time to create 1 rle with old method : 0.055962324142456055 time for calcul the mask position with numpy : 0.035652875900268555 nb_pixel_total : 21850 time to create 1 rle with old method : 0.024904727935791016 time for calcul the mask position with numpy : 0.03496837615966797 nb_pixel_total : 11689 time to create 1 rle with old method : 0.01332998275756836 time for calcul the mask position with numpy : 0.03448343276977539 nb_pixel_total : 4667 time to create 1 rle with old method : 0.005372285842895508 time for calcul the mask position with numpy : 0.03504014015197754 nb_pixel_total : 57276 time to create 1 rle with old method : 0.06479263305664062 time for calcul the mask position with numpy : 0.03602099418640137 nb_pixel_total : 40204 time to create 1 rle with old method : 0.06595659255981445 create new chi : 2.6698977947235107 time to delete rle : 0.0016260147094726562 batch 1 Loaded 23 chid ids of type : 3594 +++++++++++++++++Number RLEs to save : 7619 TO DO : save crop sub photo not yet done ! save time : 0.4637601375579834 nb_obj : 9 nb_hashtags : 3 time to prepare the origin masks : 4.731544017791748 time for calcul the mask position with numpy : 1.4404819011688232 nb_pixel_total : 6140005 time to create 1 rle with new method : 0.9289841651916504 time for calcul the mask position with numpy : 0.04311084747314453 nb_pixel_total : 248555 time to create 1 rle with new method : 1.1109180450439453 time for calcul the mask position with numpy : 0.030618906021118164 nb_pixel_total : 3936 time to create 1 rle with old method : 0.005654811859130859 time for calcul the mask position with numpy : 0.03252673149108887 nb_pixel_total : 16554 time to create 1 rle with old method : 0.020180940628051758 time for calcul the mask position with numpy : 0.03152012825012207 nb_pixel_total : 113381 time to create 1 rle with old method : 0.13720273971557617 time for calcul the mask position with numpy : 0.036468505859375 nb_pixel_total : 5715 time to create 1 rle with old method : 0.007218360900878906 time for calcul the mask position with numpy : 0.03490161895751953 nb_pixel_total : 7887 time to create 1 rle with old method : 0.010582447052001953 time for calcul the mask position with numpy : 0.03606557846069336 nb_pixel_total : 381301 time to create 1 rle with new method : 0.423816442489624 time for calcul the mask position with numpy : 0.03597068786621094 nb_pixel_total : 92105 time to create 1 rle with old method : 0.11095356941223145 time for calcul the mask position with numpy : 0.03271627426147461 nb_pixel_total : 40801 time to create 1 rle with old method : 0.045661211013793945 create new chi : 4.657726764678955 time to delete rle : 0.0017096996307373047 batch 1 Loaded 19 chid ids of type : 3594 ++++++++++++++++++Number RLEs to save : 9049 TO DO : save crop sub photo not yet done ! save time : 0.5664994716644287 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 5.113266944885254 time for calcul the mask position with numpy : 0.9367201328277588 nb_pixel_total : 6426846 time to create 1 rle with new method : 0.9270458221435547 time for calcul the mask position with numpy : 0.03379225730895996 nb_pixel_total : 38417 time to create 1 rle with old method : 0.043081045150756836 time for calcul the mask position with numpy : 0.03619790077209473 nb_pixel_total : 107395 time to create 1 rle with old method : 0.1457047462463379 time for calcul the mask position with numpy : 0.04035329818725586 nb_pixel_total : 20384 time to create 1 rle with old method : 0.02316737174987793 time for calcul the mask position with numpy : 0.03773069381713867 nb_pixel_total : 253371 time to create 1 rle with new method : 2.359842300415039 time for calcul the mask position with numpy : 0.024263858795166016 nb_pixel_total : 52990 time to create 1 rle with old method : 0.06304216384887695 time for calcul the mask position with numpy : 0.025621652603149414 nb_pixel_total : 103062 time to create 1 rle with old method : 0.13087081909179688 time for calcul the mask position with numpy : 0.023628950119018555 nb_pixel_total : 47775 time to create 1 rle with old method : 0.05391860008239746 create new chi : 4.971774339675903 time to delete rle : 0.0017399787902832031 batch 1 Loaded 15 chid ids of type : 3594 ++++++++Number RLEs to save : 7576 TO DO : save crop sub photo not yet done ! save time : 0.4714391231536865 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 4.148108720779419 time for calcul the mask position with numpy : 0.5467250347137451 nb_pixel_total : 6117969 time to create 1 rle with new method : 0.5621647834777832 time for calcul the mask position with numpy : 0.02140522003173828 nb_pixel_total : 21355 time to create 1 rle with old method : 0.025197505950927734 time for calcul the mask position with numpy : 0.02196335792541504 nb_pixel_total : 21834 time to create 1 rle with old method : 0.02568984031677246 time for calcul the mask position with numpy : 0.022992849349975586 nb_pixel_total : 24505 time to create 1 rle with old method : 0.030122756958007812 time for calcul the mask position with numpy : 0.024683237075805664 nb_pixel_total : 360877 time to create 1 rle with new method : 0.5666284561157227 time for calcul the mask position with numpy : 0.021215438842773438 nb_pixel_total : 5058 time to create 1 rle with old method : 0.005958080291748047 time for calcul the mask position with numpy : 0.020773887634277344 nb_pixel_total : 21079 time to create 1 rle with old method : 0.023758888244628906 time for calcul the mask position with numpy : 0.021693706512451172 nb_pixel_total : 26089 time to create 1 rle with old method : 0.03136897087097168 time for calcul the mask position with numpy : 0.022051572799682617 nb_pixel_total : 33757 time to create 1 rle with old method : 0.04053807258605957 time for calcul the mask position with numpy : 0.027801990509033203 nb_pixel_total : 417717 time to create 1 rle with new method : 0.5498762130737305 create new chi : 2.7017781734466553 time to delete rle : 0.00127410888671875 batch 1 Loaded 19 chid ids of type : 3594 +++++++++++++Number RLEs to save : 7444 TO DO : save crop sub photo not yet done ! save time : 0.4491569995880127 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 2.5290467739105225 time for calcul the mask position with numpy : 0.6049802303314209 nb_pixel_total : 6437668 time to create 1 rle with new method : 0.6012754440307617 time for calcul the mask position with numpy : 0.02163863182067871 nb_pixel_total : 7621 time to create 1 rle with old method : 0.00986170768737793 time for calcul the mask position with numpy : 0.023951053619384766 nb_pixel_total : 27446 time to create 1 rle with old method : 0.03098583221435547 time for calcul the mask position with numpy : 0.021692752838134766 nb_pixel_total : 10449 time to create 1 rle with old method : 0.011723518371582031 time for calcul the mask position with numpy : 0.022962093353271484 nb_pixel_total : 370317 time to create 1 rle with new method : 0.7749948501586914 time for calcul the mask position with numpy : 0.023405075073242188 nb_pixel_total : 196739 time to create 1 rle with new method : 0.41109538078308105 create new chi : 2.641247510910034 time to delete rle : 0.0008244514465332031 batch 1 Loaded 11 chid ids of type : 3594 +++++Number RLEs to save : 4620 TO DO : save crop sub photo not yet done ! save time : 0.31719183921813965 nb_obj : 10 nb_hashtags : 4 time to prepare the origin masks : 5.422961711883545 time for calcul the mask position with numpy : 0.7108020782470703 nb_pixel_total : 6185468 time to create 1 rle with new method : 1.1667613983154297 time for calcul the mask position with numpy : 0.03605031967163086 nb_pixel_total : 3264 time to create 1 rle with old method : 0.003834247589111328 time for calcul the mask position with numpy : 0.03123950958251953 nb_pixel_total : 36961 time to create 1 rle with old method : 0.04162764549255371 time for calcul the mask position with numpy : 0.02240157127380371 nb_pixel_total : 16528 time to create 1 rle with old method : 0.018865346908569336 time for calcul the mask position with numpy : 0.022943496704101562 nb_pixel_total : 10648 time to create 1 rle with old method : 0.012444496154785156 time for calcul the mask position with numpy : 0.023067712783813477 nb_pixel_total : 188076 time to create 1 rle with new method : 0.7231717109680176 time for calcul the mask position with numpy : 0.02224254608154297 nb_pixel_total : 40089 time to create 1 rle with old method : 0.05006146430969238 time for calcul the mask position with numpy : 0.02245497703552246 nb_pixel_total : 175550 time to create 1 rle with new method : 0.6322596073150635 time for calcul the mask position with numpy : 0.022307634353637695 nb_pixel_total : 16234 time to create 1 rle with old method : 0.018598079681396484 time for calcul the mask position with numpy : 0.02359628677368164 nb_pixel_total : 185535 time to create 1 rle with new method : 0.9032580852508545 time for calcul the mask position with numpy : 0.035857439041137695 nb_pixel_total : 191887 time to create 1 rle with new method : 0.4151124954223633 create new chi : 5.090061664581299 time to delete rle : 0.0028684139251708984 batch 1 Loaded 21 chid ids of type : 3594 ++++++++++++Number RLEs to save : 9823 TO DO : save crop sub photo not yet done ! save time : 0.5589923858642578 nb_obj : 11 nb_hashtags : 3 time to prepare the origin masks : 4.912927150726318 time for calcul the mask position with numpy : 0.9908154010772705 nb_pixel_total : 6116625 time to create 1 rle with new method : 0.9971818923950195 time for calcul the mask position with numpy : 0.03959512710571289 nb_pixel_total : 73536 time to create 1 rle with old method : 0.10549330711364746 time for calcul the mask position with numpy : 0.03855395317077637 nb_pixel_total : 73255 time to create 1 rle with old method : 0.09413456916809082 time for calcul the mask position with numpy : 0.03801751136779785 nb_pixel_total : 93001 time to create 1 rle with old method : 0.11039495468139648 time for calcul the mask position with numpy : 0.037120819091796875 nb_pixel_total : 50800 time to create 1 rle with old method : 0.05757617950439453 time for calcul the mask position with numpy : 0.03859376907348633 nb_pixel_total : 61141 time to create 1 rle with old method : 0.07227373123168945 time for calcul the mask position with numpy : 0.037001609802246094 nb_pixel_total : 48929 time to create 1 rle with old method : 0.05740642547607422 time for calcul the mask position with numpy : 0.03453803062438965 nb_pixel_total : 10159 time to create 1 rle with old method : 0.011718988418579102 time for calcul the mask position with numpy : 0.03778338432312012 nb_pixel_total : 6685 time to create 1 rle with old method : 0.009074926376342773 time for calcul the mask position with numpy : 0.037215471267700195 nb_pixel_total : 87629 time to create 1 rle with old method : 0.10344171524047852 time for calcul the mask position with numpy : 0.03891181945800781 nb_pixel_total : 202092 time to create 1 rle with new method : 0.8498620986938477 time for calcul the mask position with numpy : 0.03740954399108887 nb_pixel_total : 226388 time to create 1 rle with new method : 0.5604228973388672 create new chi : 4.541182518005371 time to delete rle : 0.00916743278503418 batch 1 Loaded 23 chid ids of type : 3594 ++++++++++++++Number RLEs to save : 11526 TO DO : save crop sub photo not yet done ! save time : 0.7082409858703613 map_output_result : {1338503848: (0.0, 'Should be the crop_list due to order', 0.0), 1338503842: (0.0, 'Should be the crop_list due to order', 0.0), 1338502347: (0.0, 'Should be the crop_list due to order', 0.0), 1338502244: (0.0, 'Should be the crop_list due to order', 0.0), 1338502242: (0.0, 'Should be the crop_list due to order', 0.0), 1338502132: (0.0, 'Should be the crop_list due to order', 0.0), 1338501699: (0.0, 'Should be the crop_list due to order', 0.0), 1338501695: (0.0, 'Should be the crop_list due to order', 0.0), 1338501694: (0.0, 'Should be the crop_list due to order', 0.0), 1338501691: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1338503848, 1338503842, 1338502347, 1338502244, 1338502242, 1338502132, 1338501699, 1338501695, 1338501694, 1338501691] Looping around the photos to save general results len do output : 10 /1338503848.Didn't retrieve data . /1338503842.Didn't retrieve data . /1338502347.Didn't retrieve data . /1338502244.Didn't retrieve data . /1338502242.Didn't retrieve data . /1338502132.Didn't retrieve data . /1338501699.Didn't retrieve data . /1338501695.Didn't retrieve data . /1338501694.Didn't retrieve data . /1338501691.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503848', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503842', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502347', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502244', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502242', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502132', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501699', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501695', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501694', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501691', None, None, None, None, None, '2598940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013325214385986328 save_final save missing photos in datou_result : time spend for datou_step_exec : 102.59106779098511 time spend to save output : 0.014382600784301758 total time spend for step 3 : 102.60545039176941 step4:ventilate_hashtags_in_portfolio Wed Feb 19 09:14:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20669347 get user id for portfolio 20669347 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20669347 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','papier','pet_fonce','metal','pehd','flou','autre','environnement','background','pet_clair','carton')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20669347 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','papier','pet_fonce','metal','pehd','flou','autre','environnement','background','pet_clair','carton')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20669347 AND mptpi.`type`=3594 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','papier','pet_fonce','metal','pehd','flou','autre','environnement','background','pet_clair','carton')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20669394,20669395,20669396,20669397,20669398,20669399,20669400,20669401,20669402,20669403,20669404?tags=mal_croppe,papier,pet_fonce,metal,pehd,flou,autre,environnement,background,pet_clair,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1338503848, 1338503842, 1338502347, 1338502244, 1338502242, 1338502132, 1338501699, 1338501695, 1338501694, 1338501691] Looping around the photos to save general results len do output : 1 /20669347. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503848', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503842', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502347', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502244', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502242', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502132', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501699', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501695', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501694', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501691', None, None, None, None, None, '2598940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.013929605484008789 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6383612155914307 time spend to save output : 0.014165878295898438 total time spend for step 4 : 1.652527093887329 step5:final Wed Feb 19 09:14:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1338503848: ('0.11530906749273781',), 1338503842: ('0.11530906749273781',), 1338502347: ('0.11530906749273781',), 1338502244: ('0.11530906749273781',), 1338502242: ('0.11530906749273781',), 1338502132: ('0.11530906749273781',), 1338501699: ('0.11530906749273781',), 1338501695: ('0.11530906749273781',), 1338501694: ('0.11530906749273781',), 1338501691: ('0.11530906749273781',)} new output for save of step final : {1338503848: ('0.11530906749273781',), 1338503842: ('0.11530906749273781',), 1338502347: ('0.11530906749273781',), 1338502244: ('0.11530906749273781',), 1338502242: ('0.11530906749273781',), 1338502132: ('0.11530906749273781',), 1338501699: ('0.11530906749273781',), 1338501695: ('0.11530906749273781',), 1338501694: ('0.11530906749273781',), 1338501691: ('0.11530906749273781',)} [1338503848, 1338503842, 1338502347, 1338502244, 1338502242, 1338502132, 1338501699, 1338501695, 1338501694, 1338501691] Looping around the photos to save general results len do output : 10 /1338503848.Didn't retrieve data . /1338503842.Didn't retrieve data . /1338502347.Didn't retrieve data . /1338502244.Didn't retrieve data . /1338502242.Didn't retrieve data . /1338502132.Didn't retrieve data . /1338501699.Didn't retrieve data . /1338501695.Didn't retrieve data . /1338501694.Didn't retrieve data . /1338501691.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503848', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503842', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502347', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502244', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502242', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502132', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501699', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501695', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501694', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501691', None, None, None, None, None, '2598940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013811588287353516 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11674642562866211 time spend to save output : 0.014240503311157227 total time spend for step 5 : 0.13098692893981934 step6:blur_detection Wed Feb 19 09:14:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde.jpg resize: (2160, 3264) 1338503848 -3.5801846793482532 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e.jpg resize: (2160, 3264) 1338503842 -2.6954941657764877 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394.jpg resize: (2160, 3264) 1338502347 -3.75775726775218 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58.jpg resize: (2160, 3264) 1338502244 -1.584372700663978 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d.jpg resize: (2160, 3264) 1338502242 -2.601366010911846 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249.jpg resize: (2160, 3264) 1338502132 -2.1685583949134037 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea.jpg resize: (2160, 3264) 1338501699 -5.514837263490734 treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96.jpg resize: (2160, 3264) 1338501695 -1.042368014391734 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514.jpg resize: (2160, 3264) 1338501694 -3.4976358838950814 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0.jpg resize: (2160, 3264) 1338501691 -3.8526181967216964 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025114_0.png resize: (136, 114) 1338506035 -2.981098765840073 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025112_0.png resize: (110, 138) 1338506036 -0.01786786278698234 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025109_0.png resize: (109, 157) 1338506037 -1.5336859858066247 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025111_0.png resize: (229, 198) 1338506038 -2.931561635444242 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025107_0.png resize: (306, 294) 1338506039 -2.5413362465201006 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025106_0.png resize: (190, 160) 1338506040 2.7621164963473204 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025118_0.png resize: (73, 113) 1338506041 -0.663978403252734 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025123_0.png resize: (163, 120) 1338506042 -1.4679763376984887 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025122_0.png resize: (91, 92) 1338506043 -1.0651070260764737 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025131_0.png resize: (190, 112) 1338506044 -2.2525048408713615 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025132_0.png resize: (123, 119) 1338506045 -1.9097565133562608 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025135_0.png resize: (878, 730) 1338506046 -0.6371850658038717 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025139_0.png resize: (288, 560) 1338506047 -4.471314193762435 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025144_0.png resize: (543, 360) 1338506049 -1.6194976042819307 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025143_0.png resize: (347, 340) 1338506051 -3.3118737446513284 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025138_0.png resize: (305, 236) 1338506053 -1.2564702524941185 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025136_0.png resize: (164, 201) 1338506055 -3.5419640819223623 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025145_0.png resize: (253, 469) 1338506057 -2.675385489496638 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025141_0.png resize: (65, 193) 1338506059 -2.534933661715248 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025137_0.png resize: (212, 190) 1338506061 -2.7068036022294977 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025147_0.png resize: (237, 420) 1338506063 -2.4633745482467777 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025149_0.png resize: (108, 147) 1338506065 0.12260069831102091 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025148_0.png resize: (53, 127) 1338506067 -1.7434291600851137 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025154_0.png resize: (655, 620) 1338506069 0.10868918955548297 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025153_0.png resize: (248, 346) 1338506071 0.5436752000227778 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025146_0.png resize: (174, 425) 1338506072 1.898498509451713 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025152_0.png resize: (117, 128) 1338506073 -1.512531990183449 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025151_0.png resize: (279, 264) 1338506074 -1.4458920448731425 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025150_0.png resize: (201, 161) 1338506075 -1.5735066083808755 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025165_0.png resize: (904, 644) 1338506076 0.002206728973285569 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025157_0.png resize: (278, 282) 1338506077 -1.4169637479254342 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025161_0.png resize: (71, 94) 1338506078 3.3385812534691395 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025160_0.png resize: (58, 189) 1338506079 -1.9654250446814054 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025164_0.png resize: (381, 462) 1338506080 -2.520258685098837 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025167_0.png resize: (435, 317) 1338506081 -1.2179922547599644 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025171_0.png resize: (466, 382) 1338506082 -0.7981565966829061 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025166_0.png resize: (517, 231) 1338506083 -3.3327865309906897 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025179_0.png resize: (187, 189) 1338506084 -1.7940368605072214 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025174_0.png resize: (245, 236) 1338506085 -0.786177989238384 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025177_0.png resize: (100, 75) 1338506086 -3.959114410940575 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025175_0.png resize: (232, 250) 1338506087 -3.9641517548465366 treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025184_0.png resize: (114, 121) 1338506088 -0.5773066543657882 treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025185_0.png resize: (203, 203) 1338506089 -1.3539922091677694 treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025182_0.png resize: (463, 507) 1338506090 -0.6086018692901698 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025193_0.png resize: (162, 125) 1338506091 -2.0763554298625446 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025188_0.png resize: (498, 718) 1338506092 -2.6005959598791115 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025189_0.png resize: (198, 126) 1338506093 -2.2744139116692077 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025194_0.png resize: (141, 239) 1338506094 -3.7304423377529354 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025202_0.png resize: (296, 266) 1338506095 -1.8916345522630211 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025200_0.png resize: (112, 99) 1338506096 -2.1363569090894616 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025206_0.png resize: (460, 305) 1338506097 -2.0348882009789233 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025204_0.png resize: (502, 553) 1338506098 -4.136114104859485 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025207_0.png resize: (285, 383) 1338506099 -1.474116162064855 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025203_0.png resize: (313, 242) 1338506100 1.1157606754391836 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025201_0.png resize: (315, 118) 1338506101 -2.8433472225747507 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025110_0.png resize: (413, 160) 1338506103 -3.007418814159046 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025115_0.png resize: (288, 256) 1338506105 -2.7780947258336406 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025120_0.png resize: (178, 173) 1338506106 -1.4894361257519233 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025117_0.png resize: (493, 356) 1338506107 -2.5491582244154825 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025116_0.png resize: (180, 238) 1338506108 -2.1492512539955864 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025121_0.png resize: (81, 197) 1338506109 -1.6292241804953966 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025124_0.png resize: (125, 246) 1338506110 -1.7477371720844532 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025128_0.png resize: (266, 343) 1338506111 -2.3582990594304483 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025119_0.png resize: (195, 181) 1338506112 -2.793994586876236 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025140_0.png resize: (184, 161) 1338506113 -2.9346033859753446 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025129_0.png resize: (421, 312) 1338506114 -1.4075852329204992 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025133_0.png resize: (115, 172) 1338506115 -2.5463151533377286 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025155_0.png resize: (66, 36) 1338506116 -3.5303979364474554 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025163_0.png resize: (120, 243) 1338506117 -3.276906408219934 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025168_0.png resize: (337, 230) 1338506118 -0.6805317043322292 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025170_0.png resize: (184, 170) 1338506119 -2.657060415684044 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025172_0.png resize: (154, 369) 1338506120 -4.042537901184557 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025173_0.png resize: (668, 887) 1338506121 -5.986731003750181 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025176_0.png resize: (176, 187) 1338506122 -5.3249951063927465 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025187_0.png resize: (541, 1105) 1338506123 0.052757745632548 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025197_0.png resize: (744, 568) 1338506124 -3.6413168902089588 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025205_0.png resize: (493, 288) 1338506125 -2.673893078748147 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025181_0.png resize: (210, 169) 1338506126 -5.076489011214961 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025180_0.png resize: (171, 158) 1338506127 -5.267283551985416 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025108_0.png resize: (288, 1648) 1338506142 -2.234207514792188 treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025113_0.png resize: (347, 576) 1338506143 -2.848480214768471 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025125_0.png resize: (415, 293) 1338506144 -1.4456097032117512 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025126_0.png resize: (281, 1637) 1338506145 -2.248772088105883 treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025127_0.png resize: (305, 446) 1338506146 -3.875227044650041 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025134_0.png resize: (442, 183) 1338506147 -3.4665972110942267 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025130_0.png resize: (294, 1756) 1338506148 -0.4484255672561767 treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025142_0.png resize: (265, 358) 1338506149 -3.0515363050849986 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025159_0.png resize: (326, 1679) 1338506150 -1.407741095849368 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025158_0.png resize: (482, 329) 1338506152 -2.77935656865437 treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025162_0.png resize: (483, 451) 1338506153 -2.1667135370895214 treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025169_0.png resize: (602, 609) 1338506154 -3.414199527189911 treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025178_0.png resize: (335, 1554) 1338506155 -1.4212045264448094 treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025183_0.png resize: (284, 1854) 1338506156 -1.1098416514615754 treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025186_0.png resize: (128, 87) 1338506157 -4.025862764362529 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025192_0.png resize: (742, 415) 1338506158 -3.2644830125790865 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025195_0.png resize: (309, 228) 1338506159 -2.8466007442650714 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025196_0.png resize: (107, 131) 1338506160 -2.5094011332072967 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025190_0.png resize: (464, 504) 1338506161 -4.2002811726206275 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025199_0.png resize: (186, 720) 1338506162 -2.0400788540441663 treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025198_0.png resize: (952, 278) 1338506163 -2.0513746359050127 treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025191_0.png resize: (359, 203) 1338506165 -3.6318671205792703 treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025156_0.png resize: (146, 247) 1338506167 -1.621055654324417 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 112 time used for this insertion : 0.16742706298828125 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 112 time used for this insertion : 0.9993977546691895 save missing photos in datou_result : time spend for datou_step_exec : 42.524263858795166 time spend to save output : 1.1731972694396973 total time spend for step 6 : 43.69746112823486 step7:brightness Wed Feb 19 09:15:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde.jpg treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e.jpg treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394.jpg treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58.jpg treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d.jpg treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249.jpg treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea.jpg treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96.jpg treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514.jpg treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0.jpg treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025114_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025112_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025109_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025111_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025107_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025106_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025118_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025123_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025122_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025131_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025132_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025135_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025139_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025144_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025143_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025138_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025136_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025145_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025141_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025137_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025147_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025149_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025148_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025154_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025153_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025146_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025152_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025151_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025150_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025165_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025157_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025161_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025160_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025164_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025167_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025171_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025166_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025179_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025174_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025177_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025175_0.png treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025184_0.png treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025185_0.png treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025182_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025193_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025188_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025189_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025194_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025202_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025200_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025206_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025204_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025207_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025203_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025201_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025110_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025115_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025120_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025117_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025116_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025121_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025124_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025128_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025119_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025140_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025129_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025133_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025155_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025163_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025168_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025170_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025172_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025173_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025176_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025187_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025197_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025205_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025181_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025180_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025108_0.png treat image : temp/1739952630_3386087_1338503848_4164c74d19faabafc23d685bfc065bde_rle_crop_3680025113_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025125_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025126_0.png treat image : temp/1739952630_3386087_1338503842_a5898fa3fd57f71e327d456d41d72e0e_rle_crop_3680025127_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025134_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025130_0.png treat image : temp/1739952630_3386087_1338502347_a30107689b68e15e53c41b517eeb3394_rle_crop_3680025142_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025159_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025158_0.png treat image : temp/1739952630_3386087_1338502242_b896896deba2d3946c9c6d54b47a482d_rle_crop_3680025162_0.png treat image : temp/1739952630_3386087_1338502132_9b3197df582e4f9b9168607ac8afb249_rle_crop_3680025169_0.png treat image : temp/1739952630_3386087_1338501699_4d9dedd5315b9e5f7a18f6c78ef017ea_rle_crop_3680025178_0.png treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025183_0.png treat image : temp/1739952630_3386087_1338501695_44e47c7289cebadc9381640f149d7d96_rle_crop_3680025186_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025192_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025195_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025196_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025190_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025199_0.png treat image : temp/1739952630_3386087_1338501691_844357d95f8d69ca258ae782c7c8cba0_rle_crop_3680025198_0.png treat image : temp/1739952630_3386087_1338501694_fc530f3b4040d1616c4d0d1d217cb514_rle_crop_3680025191_0.png treat image : temp/1739952630_3386087_1338502244_c690b59cbfeaa5c08b1c663f04f2ed58_rle_crop_3680025156_0.png Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 112 time used for this insertion : 0.2613034248352051 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 112 time used for this insertion : 0.12018036842346191 save missing photos in datou_result : time spend for datou_step_exec : 10.433544158935547 time spend to save output : 0.3871948719024658 total time spend for step 7 : 10.820739030838013 step8:velours_tree Wed Feb 19 09:15:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.24228858947753906 time spend to save output : 6.008148193359375e-05 total time spend for step 8 : 0.24234867095947266 step9:send_mail_cod Wed Feb 19 09:15:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20669347_19-02-2025_09_15_42.pdf 20669394 imagette206693941739952942 20669395 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206693951739952942 20669396 change filename to text .imagette206693961739952943 20669397 change filename to text .change filename to text .imagette206693971739952943 20669398 imagette206693981739952943 20669399 imagette206693991739952943 20669400 change filename to text .imagette206694001739952943 20669402 imagette206694021739952943 20669403 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206694031739952943 20669404 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206694041739952945 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20669347 and hashtag_type = 3594 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20669394,20669395,20669396,20669397,20669398,20669399,20669400,20669401,20669402,20669403,20669404?tags=mal_croppe,papier,pet_fonce,metal,pehd,flou,autre,environnement,background,pet_clair,carton args[1338503848] : ((1338503848, -3.5801846793482532, 492609224), (1338503848, 0.011295274665998185, 2107752395), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338503842] : ((1338503842, -2.6954941657764877, 492609224), (1338503842, -0.29969530192322813, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338502347] : ((1338502347, -3.75775726775218, 492609224), (1338502347, -0.3832278283382887, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338502244] : ((1338502244, -1.584372700663978, 492688767), (1338502244, -0.32273429371293694, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338502242] : ((1338502242, -2.601366010911846, 492609224), (1338502242, -0.43754696933376347, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338502132] : ((1338502132, -2.1685583949134037, 492609224), (1338502132, -0.231082535954971, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338501699] : ((1338501699, -5.514837263490734, 492609224), (1338501699, -0.42250682732242795, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338501695] : ((1338501695, -1.042368014391734, 492688767), (1338501695, -0.28207071806812295, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338501694] : ((1338501694, -3.4976358838950814, 492609224), (1338501694, -0.2976289960029555, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com args[1338501691] : ((1338501691, -3.8526181967216964, 492609224), (1338501691, -0.36535201419370317, 496442774), '0.11530906749273781') We are sending mail with results at report@fotonower.com refus_total : 0.11530906749273781 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20669347 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 error for generate pdf : 'mal_croppe' message_in_mail: Bonjour,
Veuillez trouver ci dessous les résultats du service carac on demand pour le portfolio: https://www.fotonower.com/view/20669347

https://www.fotonower.com/image?json=false&list_photos_id=1338503848
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338503842
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338502347
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338502244
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338502242
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338502132
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338501699
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338501695
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338501694
Bravo, la photo est bien prise.
https://www.fotonower.com/image?json=false&list_photos_id=1338501691
Bravo, la photo est bien prise.

Dans ces conditions,le taux de refus est: 11.53%
Veuillez trouver les photos des contaminants.

exemples de contaminants: papier: https://www.fotonower.com/view/20669395?limit=200
exemples de contaminants: pet_fonce: https://www.fotonower.com/view/20669396?limit=200
exemples de contaminants: metal: https://www.fotonower.com/view/20669397?limit=200
exemples de contaminants: autre: https://www.fotonower.com/view/20669400?limit=200
exemples de contaminants: pet_clair: https://www.fotonower.com/view/20669403?limit=200
exemples de contaminants: carton: https://www.fotonower.com/view/20669404?limit=200
Lien vers velours :https://www.fotonower.com/velours/20669394,20669395,20669396,20669397,20669398,20669399,20669400,20669401,20669402,20669403,20669404?tags=mal_croppe,papier,pet_fonce,metal,pehd,flou,autre,environnement,background,pet_clair,carton.


L'équipe Fotonower 202 b'' Server: nginx Date: Wed, 19 Feb 2025 08:15:48 GMT Content-Length: 0 Connection: close X-Message-Id: 1NyL1OrwTwWvj2KBajD_qQ Access-Control-Allow-Origin: https://sendgrid.api-docs.io Access-Control-Allow-Methods: POST Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl Access-Control-Max-Age: 600 X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html Strict-Transport-Security: max-age=31536000; includeSubDomains Content-Security-Policy: frame-ancestors 'none' Cache-Control: no-cache X-Content-Type-Options: no-sniff Referrer-Policy: strict-origin-when-cross-origin Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1338503848, 1338503842, 1338502347, 1338502244, 1338502242, 1338502132, 1338501699, 1338501695, 1338501694, 1338501691] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503848', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503842', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502347', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502244', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502242', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502132', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501699', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501695', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501694', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501691', None, None, None, None, None, '2598940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.020305633544921875 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.090055227279663 time spend to save output : 0.020541667938232422 total time spend for step 9 : 6.1105968952178955 step10:split_time_score Wed Feb 19 09:15:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 10),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 19022025 20669347 Nombre de photos uploadées : 10 / 23040 (0%) 19022025 20669347 Nombre de photos taguées (types de déchets): 0 / 10 (0%) 19022025 20669347 Nombre de photos taguées (volume) : 0 / 10 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 7.3909759521484375e-06 ?????????? elapsed_time : fill_and_build_computed_from_old_data 0.0005316734313964844 elapsed_time : insert_dashboard_record_day_entry 0.022508859634399414 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20669340 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20669342 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20669344 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20669346 order by id desc limit 1 Qualite : 0.11530906749273781 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20669347 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20669347 AND mptpi.`type`=3594 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'19022025': {'nb_upload': 10, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1338503848, 1338503842, 1338502347, 1338502244, 1338502242, 1338502132, 1338501699, 1338501695, 1338501694, 1338501691] Looping around the photos to save general results len do output : 1 /20669347Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503848', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338503842', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502347', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502244', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502242', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338502132', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501699', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501695', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501694', None, None, None, None, None, '2598940') ('3318', None, None, None, None, None, None, None, '2598940') ('3318', '20669347', '1338501691', None, None, None, None, None, '2598940') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.016908645629882812 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.051684379577637 time spend to save output : 0.01719808578491211 total time spend for step 10 : 4.068882465362549 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 10 set_done_treatment 138.54user 141.16system 5:26.55elapsed 85%CPU (0avgtext+0avgdata 4298408maxresident)k 774272inputs+109088outputs (9526major+12114991minor)pagefaults 0swaps